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Introduction: 


This  project  serves  to  address  the  urgent  need  for  marked  improvement  in  prostate  cancer  (PCa) 
detection  and  diagnosis  via  the  use  of  multimodal  imaging  for  (a)  biopsy  planning  and 
guidance,  (b)  PCa  grading  and  staging,  (c)  active  surveillance,  and  (d)  treatment  targeting  and 
monitoring;  due  to  severe  limitations  with  current  clinical  protocol.  For  example,  in  May  2012 
the  US  Preventive  Task  Force  issued  a  recommendation  against  PSA  based  screening  for  PCa, 
concluding  that  it  causes  more  harm  than  good.  Current  prostate  biopsy  procedures  which 
primarily  use  conventional  ultrasound  (US)  only  have  an  estimated  sensitivity  of  approximately 
50%,  with  90%  of  US-guided  biopsies  being  negative.  Similarly,  focal  therapy  to  treat  PCa  is 
gaining  interest  as  a  mean  of  reducing  toxic  side  effects,  but  it  is  unlikely  to  gain  favor  unless  a 
reliable  mean  of  imaging  and  subsequently  validating  presence  and  extent  of  PCa  is 
forthcoming. 

A  novel  elastography  tool,  Prostate  Mechanical  Imaging  (PMI)  developed  by  Artann  Labs,  is 
newly  FDA  approved  for  PCa  screening.  Unlike  conventional  B-mode  US,  PMI  allows  for  the 
measurement  of  gland  volume  and  mechanical  stress  patterns  on  the  gland  surface  through  the 
rectal  wall  with  pressure  sensor  arrays.  However,  PMI  lacks  the  high  spatial  resolution  of 
Magnetic  Resonance  Imaging  (MRI).  Multi-Parametric  MRI  (T2W,  T1W,  Diffusion,  and  DCE) 
has  recently  shown  great  promise  for  improving  screening  and  detection  of  PCa  as  it  provides 
significant  structural  and  functional  parameters  for  disease  characterization.  This  project  is  to 
develop  novel  computerized  fusion  methods  for  precise  registration  of  MRI  and  PMI  in  order  to 
enable  improved  PCa  detection  in  vivo  compared  to  using  either  PMI  or  MRI  alone.  This 
research  will  have  an  impact  on  development  of  reliable  and  practical  clinical  means  of  imaging 
PCa  for  improved  (1)  biopsy  planning  and  guidance,  (2)  grading  and  prognosis  formulation,  (3) 
active  surveillance,  (4)  treatment  planning,  (5)  disease  targeting,  and  (6)  treatment  monitoring. 
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Accomplishments 
Summary  of  Progress  to  Date: 

The  project  has  resulted  in  3  peer  reviewed  abstracts,  1  ongoing  journal  article,  and  1  approved 
patent.  We  are  currently  on  track  to  complete  all  the  remaining  tasks  proposed  in  this  project  by 
the  end  of  the  project  period.  Below,  we  describe  the  specific  progress  performed  under  each  of 
the  original  specific  aims  and  also  describe  the  pending  tasks  for  each  specific  aim. 

Milestone  supposed  to  achieve  in  the  12th  month 

Conference  such  as  SPIE  Medical  Imaging  or  International  Symposium  in  Biomedical 
Imaging 

o  Three  abstracts  were  accepted  on 

1-  24th  International  Society  for  Magnetic  Resonance  Medicine  (ISMRM),  7-13 
May  2016,  Singapore. 

2-  American  Society  of  Clinical  Oncology  (ASCO)  Annual  Meeting,  Jum  3-7 
2016,  Chicago,  IL. 

3-  102nd  Scientific  Assembly  and  Annual  Meeting  of  Radiological  Society  of 
North  America  (RSNA),  Nov  27-  Dec  3,  Chicago,  IL. 

A  clinical  imaging  analysis  journal  such  as  Journal  of  Urology. 

o  A  journal  on  evaluating  the  computer  extracted  radiomics  for  discriminating  the 
cancer  from  benign  infection  (Specific  Aim  2,  Task  1,2  and  3)  submitted  to 
Nature  Scientific  Report 

o  A  journal  paper  is  in  preparation  on  using  the  statistical  shape  model  for  the 
prostate  segmentation  (Specific  Aim  1,  Task  1) 

Registration  techniques  to  align  MRI,  PMI,  histology. 

o  An  algorithm  is  developed  and  presented  on  24th  International  Society  for 
Magnetic  Resonance  Medicine  (ISMRM),  7-13  May  2016,  Singapore. 

Methods  to  map  cancer  extent  from  histology  (ground  truth)  onto  radiology/mechanical 
imaging  data  to  define  signatures  on  in  vivo  imaging 

o  An  algorithm  is  developed  and  presented  on  24th  International  Society  for 
Magnetic  Resonance  Medicine  (ISMRM),  7-13  May  2016,  Singapore. 

Fusion  and  integrated  visualization  of  mechanical  and  multi-parametric  imaging  data, 
o  American  Society  of  Clinical  Oncology  (ASCO)  Annual  Meeting,  Jum  3-7  2016, 
Chicago,  IL. 

o  102nd  Scientific  Assembly  and  Annual  Meeting  of  Radiological  Society  of  North 
America  (RSNA),  Nov  27-  Dec  3,  Chicago,  IL. 
o  An  algorithm  is  developed  and  presented  on  24th  International  Society  for 
Magnetic  Resonance  Medicine  (ISMRM),  7-13  May  2016,  Singapore. 
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Research-Specific  Major  Tast  1:  Employing  co-registration  tools  to  align  MP-MRI,  PMI, 
ex  vivo  histologic  sections 

•  Aim  1-Task  1:  Pre-processing  of  MP-MRI  and  histology,  Acquisition  of  multi -protocol  in 
vivo  MRI  prostate  data  (T2w,  DCE,  DWI),  prostate  mechanical  imaging  data,  with  corresponding 
whole-mount  histological  (WMH)  data  from  our  collaborators. 

My  co-mentor,  Dr.  Lee  Ponsky,  Urologist  and  Professor  at  the  University  Hospital  Cleveland 
Medical  Center  (UHCMC),  Case  Western  Reserve  University  accepted  the  primary 
responsibility  to  select  cases  for  this  project.  Besides,  Dr.  Nicolas  Bloch  in  Boston  University 
School  of  Medicine,  Boston,  Massachusetts  helped  us  by  providing  MRI  and  ultrasound  of  the 
prostate  images.  To  develop  the  co-registration  methodology  for  MRI- PMI  fusion,  we  employed 
MRI-TRUS  fusion  algorithm.  Table  1  shows  the  data  we  used  in  MRI-TRUS-WMH  fusion. 


Table  1:  Acquired  data  for  M 

[RI-TRUS-W] 

V1H  registration 

Modality 

MRIT1 

MRI  T2 

Transrectal 

Ultrasound 

Whole  Mount 
Histological 

Number  of  Studies 

12 

15 

31 

6 

The  main  part  of  pre-processing  of  the  acquired  data  was  focused  on  the  prostate  segmentation  in 
the  transrectal  ultrasound.  PI  employed  the  statistical  shape  model  for  the  prostate  segmentation. 
A  brief  description  and  the  results  of  employed  method  are  as  follows.  The  ongoing  paper  is 
attached  in  the  appendices. 


•  Using  statistical  shape  model  for  the  prostate  segmentation  in  the  transrectal  ultrasound 
image 

An  accurate  detection  of  the  prostate  volume  and  boundary  is  influential  on  diagnosis,  treatment, 
and  follow  up  of  CaP.  Within  this  framework,  prostate  segmentation  on  TRUS  imagery  is 
performed  via  introduced  spatial  statistic  aware  segmentation  paradigm.  The  spatial  prior 
probability  is  calculated  in  the  training  phase,  and  is  used  to  estimate  the  texture  feature 
parameters  corresponding  to  the  prostate  and  the  background.  Estimated  parameters  is  employed 
to  represent  an  alternative  probabilistic  presentation  of  the  prostate  in  TRUS.  Results  show  the 
3D  prostate  capsule  is  more  pronounced  in  the  new  representation  which  ultimately  results  in 
more  accurate  segmentation.  Modified  active  shape  model  (ASM)  is  introduced  and  applied  in 
the  new  3D  TURS  representation  for  prostate  segmentation.  Figure  1  demonstrates  the 
framework  of  the  employed  method. 

Prostate  segmentation  in  TRUS  is  a  challenging  task  mainly  due  to  the  high  inherent  noise  of  the 
ultrasound  images.  On  the  other  side,  the  proficiency  of  the  texture  features  in  the  prostate 
segmentation  is  already  investigated.  We  employed  the  power  of  the  texture  features  in 
presenting  the  prostate  in  the  ultrasound  images  and  also  the  spatial  statistics  information  to 
introduce  an  alternative  representation.  Figure  2  demonstrates  the  heatmap  of  the  prostate  and 
the  background  for  Haralick  features  of  the  TRUS  image,  in  which  the  prostate  capsule  is  more 
pronounced  in  compare  to  the  original  TRUS  image.  We  named  it  the  TRUS-HeatMap 
Representation. 
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Figure  1:  Schematic  of  four  steps  used  for  generating  the  foreground  and  background  heat-map 
per  texture  feature. 


Figure  2.  The  probability  heatmap  of  (a)  the  foreground  probability  and  (b)  the  background 
probability. 

•  Method  used  for  PMI  images: 

Prior  to  each  PMI  examination  the  probe  is  automatically  calibrated  by  placing  it  into  the 
calibrator.  After  this,  a  protective  disposable  sheath  is  placed  over  the  PMI  transrectal  probe 
including  the  probe  handle.  A  patient  is  asked  to  bend  at  a  90-degree  angle  at  the  hip  for  the  PMI 
examination.  The  calibrated  PMI  probe  covered  by  the  sheath  is  lubricated  and  gently  inserted 
into  the  anal  canal  with  sensor  surface  facing  downward.  The  probe  is  then  gently  inserted 
beyond  the  anal  canal  and  into  rectum  until  the  axial  projection  of  the  prostate  is  visualized  on 


8 


the  computer  screen.  The  prostate  scan  is  performed  through  a  set  of  multiple  manual 
compressions.  The  pressure  sensor  array,  in  response  to  input  applied  pressure,  produces  a 
pressure  response  map  on  a  rectal  wall  covering  the  prostate,  analogous  to  that  sensed  by  the 
physician’s  finger  palpating  the  prostate  during  DRE.  The  PMI  provides  3-D  pressure  mapping 
of  the  prostate.  This  3-D  prostate  map/image  may  be  visualized  and  analyzed  after  the 
examination.  Figure  3A  demonstrates  the  real  time  cross-sectional  images  of  the  prostate 
obtained  by  PMI,  and  3B  is  an  actual  prostate  image  of  a  patient  obtained  by  PMI. 


Distance,  mm 


Figure  3.  (A)  Real  time  cross-sectional  images  of  the  prostate  of  a  patient  obtained  by  5 
successive  pressings  of  the  probe  over  the  prostate  at  different  location  as  shown  in  the  picture. 
Patent  data:  PSA  -  7.9;  DRE  -  medium  size  asymmetrical  prostate,  left  sided  nodule,  firm, 
immobile.  PMI  findings:  hard  nodule  located  at  the  prostate  base  in  the  left  lobe.  (B)  An  example 
of  mechanical  image  of  prostate  of  a  patient. 


•  Aim  1,  Task  2:  Co-registration  of  PMI  with  MP-MRI 

Subtask  1 :  Obtain  a  segmentation  of  the  prostate  on  MP-MRI  using  the  methodology  Extract  a 

set  of  features  from  the  PMI  capable  of  distinguishing  between  prostate  tissue  and  non-prostate 

tissue. 


The  ROI  was  segmented  in  the  previous  step.  A  total  of  645  2D  texture  and  24  3D-shape  features 
were  extracted  from  the  segmented  area.  Texture  features  were  extracted  in  2D  instead  of  3D, 
since  the  available  retrospective  volumes  were  all  anisotropic.  After  extracting  per  voxel  based 
features  within  the  nodule  of  interest,  five  statistics  relating  to  mean,  variance,  minimum, 
maximum  and  the  entropy  were  calculated.  All  feature  calculations  were  implemented  using 
MATLAB®  2014b  platform  (Mathworks,  Natick,  MA).  The  description  of  the  extracted  texture 
features  and  shape  features  can  be  found  in  Table  2  and  Table  3. 
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Table  2:  Texture  features  evaluated  in  this  work. 


Feature  category 

Descriptor 

Intuitive  Description 

Haralick  features 
(Repeated  occurrence  of  grey 
level  configuration  in  the 
texture  represented  via  the 
grey-level  co-occurrence 
matrix  (GLCM),  which  varies 
rapidly  with  distance  in  fine 
textures  and  slowly  in  large 
textures) 

Inverse  Difference 
Moment  (IDM) 

IDM  is  a  reflection  of  the  presence  or 
absence  of  uniformity,  and  hence  is  a 
measure  of  local  regions  of  homogeneity 
High  IDM:  Higher  presence  of  locally 
uniform  windows  in  GLCM 

Low  IDM:  Higher  presence  of  locally 
heterogeneous  windows  in  GLCM 

Correlation 

Quantifies  the  linear  patterns  in  an  image 
based  on  the  distance  parameter. 

Sum  Entropy 

Measure  of  GLCM  relationship  to 
distribution  of  intensity  with  respect  to 
entropy.  Entropy  is  the  measure  of 
disorder. 

Sum  Variance 

Measure  of  GLCM  relationship  to 
distribution  of  intensity  with  respect  to 
variance 

High  sum  variance:  greater  standard 
deviation  of  sum  average 

Low  sum  variance:  low  standard  deviation 
of  sum  average 

Laws  features 

E5,  L5,  S5,W5,R5 
(combination  in 
both  X  and  Y 
directions) 

E-  Edges 

L-  Level 

S-  Spots 

W-  Wave 

R-  Ripple 

Laplacian  pyramids 

Multi-resolution  filters  capture  edges  at 
different  levels 

Gray  level  features 

The  basic,  intensity  based  features 
including  mean,  median,  range  and 
standard  deviation. 

Gabor  Features 

Oriented  textures  via  changes  in  direction 
and  scale;  capture  microarchitectures 

Gradient  Features 

Represent  the  directional  change  in  the 
intensity  values  of  pixels  in  the  ROI 

Local  Binary  Pattern 

Thresolding  the  window  with  the  center 
pixel  value. 
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Table  3:  Shape  features  evaluated  in  this  work. 


Features 

Description 

Size 

Including  Width,  Height,  Depth  of  bounding  box 

Area 

from  2D  slices  of  each  nodule 

Perimeter 

from  2D  slices  of  each  nodule 

Eccentricity 

foci  of  the  ellipse  and  to  major  axis  length 

Extend 

ratio  of  pixels  in  the  region  to  pixels  in  the  total  bounding  box 

Compactness 

ratio  of  the  perimeter  squared  to  the  product  of  4n  and  area 

Radial  distance 

distances  from  center  of  each  slice  to  contour  points 

Roughness 

perimeter  of  slices  divided  by  convex  perimeter 

Elongation 

from  major  and  minor  axis 

Convexity 

from  convex  hull 

Equivalent 

Diameter 

Diameter  of  circle  with  same  area  of  slices 

Sphericity 

3D  compactness 

Subtask  2  and  3:  Calculate  a  probabilistic  model  to  estimate  the  prostate  location  on  PMI  and 
Apply  an  elastic  registration  to  maximize  the  overlap  between  the  MP-MRI  segmentation  and  the 
PMI  probabilistic  model. 

Elastic  registration  methods  developed  by  the  mentor  of  the  PI,  at  Case  Western  Reserve 
University,  yield  a  mapping  of  prostate  cancer  extent  from  ex  vivo  pathology  on  to  corresponding 
MP-MRI,  by  allowing  for  recovery  of  non-linear  deformations  between  the  template  and  target 
images.  PI  extended  our  previous  approaches  by  inclusion  of  additional  imaging  data  to  facilitate 
the  registration:  (1)  block  face  photos  of  the  ex  vivo  pathology,  and  (2)  ex  vivo  MRI  of  the 
excised  RP.  The  main  steps  are: 

Module  1:  Reconstruct  the  3D  volume  of  the  histopathology  specimen,  Path3D.  This  was 
involved  (a)  aligning  the  WMH  slices  to  the  block  face  section  photos  using  2D 
deformable  registration,  (2)  applying  groupwise  registration  to  align  block  face  pictures 
to  each  other;  (3)  implicitly  reconstructing  the  3D  volume  and  providing  a  3D  volume  for 
the  tumor  (see  Figure  4). 

Module  2:  3D  histology,  Path3D,  is  aligned  to  the  ex  vivo  MRI  using  3D  deformable  co¬ 
registration.  We  then  co-registered  ex  vivo  to  in  vivo  MRI  using  3D  deformable 
registration  methods,  such  as  finite  element  modeling  (FEM).  The  transformation  was 
constrained  to  physically  meaningful  deformations.  Our  approach  determined  optimal 
deformation  forces  that  maximize  similarity  metrics  based  on  MRI  intensities.  This  will 
allow  for  mapping  of  CaP  annotation  from  histology  on  in  vivo  MRI.  Figure  4  shows  the 
result  of  registering  WMH  (IHC  and  H&E)  to  in  vivo  MRI  via  this  technique. 
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Figure  4:  (1)  3D  Reconstruction  of  ex  vivo 
pathology  via  alignment  with  block-face  photos; 

(2)  Mapping  CaP  extent  from  pathology  onto  MRI; 

(3)  PMI  -  MRI  fusion  to  map  CaP  onto  PMI. 


Module  1 :  Histology  3D  reconstruction 


Module  2:  Histology  -  MRI  fusion 


Module  3:  PMI  -  MRI  fusion 


•  Aim  1,  Task  3:  Evaluating  PMI/MP-MRI/Histology  registration. 

To  provide  an  optimize  tool  for  MRI/PMI/WMH  registration,  PI  developed  M  R I/TR  US/WM  H 
registration  algorithm.  The  method  was  published  and  presented  in  the  24th  International  Society 
for  Magnetic  Resonance  Medicine  (ISMRM),  7-13  May  2016,  Singapore.  The  published  abstract 
and  the  presented  poster  is  attached  in  the  appendices. 

Our  study  design  comprised  12  2D  planar  images  obtained  from  the  MRI  and  US  scans  of  3 
patients,  ah  of  whom  had  biopsy  confirmed  prostate  cancer  and  scheduled  for  a  radical 
prostatectomy.  A  3D  B-mode  ultrasound  scan  was  performed  followed  by  a  3  Tesla  MRI  prior  to 
surgery.  Following  surgery  and  histologic  sectioning  of  the  gland  via  a  microtome,  the  H&E 
stained  whole  mount  histologic  (WMH)  sections  were  digitized  via  a  whole  slide  scanner  and  the 
regions  of  cancer  annotated  by  an  expert  pathologist.  Deformable  co-registration  methods  were 
used  to  spatially  align  the  in  vivo  MRI,  TRUS,  and  ex  vivo  histology.  In  particular,  we  used  fully 
automatic  Multiattribute  probabilistic  prostate  elastic  registration  (MAPPER)  approach  to  fusion 
of  ultrasound  and  MRI.  We  also  manually  delineated  corresponding  landmarks  between  MRI 
and  WMH  for  deformable  co-registration  of  WMH  to  MRI.  A  total  of  129  image  features 
including  Haralick,  Gabor,  Law,  LBP,  Laplacian  features  were  extracted  from  both  the  prostate 
MRI  and  TRUS.  Each  of  the  computer  extracted  MRI  and  ultrasound  features  were  then  ranked 
via  the  Fisher  criteria  to  identify  the  features  that  best  identified  the  region  of  cancer.  Figure  5 
illustrates  the  MRI-TRUS-WMH  registration  and  mapping  of  the  cancer  extent  on  MRI  and 
TRUS. 

The  top  3  features  for  each  modality  and  corresponding  Fisher  criteria  values  are  shown  in  Table 
4.  The  classification  is  per  region  of  interest  (ROI),  i.e.  the  texture  features  for  the  cancerous  part 
of  the  prostate  is  compared  to  the  texture  features  of  the  noncancerous  confounding  regions.  Top 
three  texture  features,  contrast  variance,  contrast  entropy,  and  contrast  inverse  moment  were 
selected  by  the  theoretical  linear  discriminant  analysis  (LDA)  classifier  for  MRI  and  yielded  an 
area  under  the  receiver  operating  characteristic  curve  (AUC)  of  0.83,  0.77  and  0.70  for 
identifying  cancerous  ROIs  in  MRI.  By  comparison,  top  three  most  predictive  features  identified 
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for  TRUS  were  contrast  inverse  moment,  contrast  variance,  and  contrast  entropy.  These  features 
yielded  an  AUC  of  0.75,  0.69,  and  0.66,  respectively.  By  combining  the  top  two  texture  features 
on  MRI  and  the  most  informative  texture  feature  on  TRUS,  the  LDA  based  predictor  yielded  an 
AUC  of  0.88  in  predicting  presence  of  prostate  cancer.  Figure  6  illustrates  the  scatter  plot  of  the 
prostate  cancer  versus  the  non-cancer  cases  in  three  dimensional  most  informative  texture  feature 
space. 


Figure  5:  Registration  of  MRI,  TRUS  and  WMH:  Two  2D  planar  images  of  (a),(g)  WMH  and  (b),(h) 
corresponding  MRI.  (c),(i)  WMH  and  MRI  checkerboard  overlays  showing  alignment  between  the  two  modalities. 
(d),(j)  MRI  with  cancer  annotation  obtained  from  WMH  (green).  (e),(k)  TRUS  with  cancer  annotation  obtained 
from  WMH  (green).  (f),(l)  Fused  MRI-TRUS  images  shown  as  checkerboards  with  cancer  annotation  obtained  from 
WMH  (green). 
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Figure  6:  Scatter  plot  of  three  most  discriminative  texture  features 


Table  4:  Top  3  features  of  MRI  and  TRUS,  and  their  AUC  values 

Features  for  MRI  Contrast  Variance  Contrast 

Entropy 

AUC _ 083 _ 0.77 

Features  for  TRUS  Contrast.  Inverse  Moment  Contrast 

Variance 

AUC  0.75  0.69 


Contrast. 

Inverse 

Moment 

0.70 

Contrast 

Entropy 

0.66 


Training  Specific  Aims:  Training  and  educational  development  in  prostate  cancer 
research. 

Subtask  1\  Mechanical  Imaging  Training  session  in  Artann  Labs  (Two  weeks  in  the  first  year  and 
two  weeks  in  the  second  year).  The  PI  scheduled  the  first  training  session  for  the  middle  of 
December  2016. 

Subtask  2:  Seminars 

PI  had  weekly  mentor  lab  meeting,  in  the  Center  for  Computational  Imaging  and 
Personalized  Diagnostics  (CCIPD)  at  Case  Western  Reserve  University. 

PI  is  participating  weekly  Prostate  Imaging  Reporting  and  Data  System  meeting  in  the 
Department  of  Radiology,  Case  Medical  Center  University  Hospital  with  Pi’s  mentor, 
clinical  co-mentor  and  radiology  collaborators. 

PI  is  participating  the  Biomedical  Image  Analysis  Literary  Guild  (LG)  seminar,  Case 
Western  Reserve  University.  Hosted  by  Pi’s  mentor. 

PI  is  participating  the  majority  of  Imaging  Hour  meeting,  Department  of  Biomedical 
Engineering,  Case  Western  Reserve  University. 
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PI  is  participating  the  majority  of  monthly  Cancer  Center  Seminar  Series,  Case 
Comprehensive  Cancer  Center. 

Subtask  3:  Course  works 

PI  studied  the  course  materials  of  “Cancer  Biology,  Immunology,  and  Pathology”,  and 
“Biostatistics”. 

Subtask  4:  Attending  a  scientific  meeting  in  relevant  scientific  field 
PI  participated  and  also  presented  his  works  on 

o  American  Society  of  Clinical  Oncology  (ASCO)  Annual  Meeting,  Jum  3-7  2016, 
Chicago,  IL. 

o  102nd  Scientific  Assembly  and  Annual  Meeting  of  Radiological  Society  of  North 
America  (RSNA),  Nov  27-  Dec  3,  Chicago,  IL. 
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Impact 


A  novel  elastography  tool,  Prostate  Mechanical  Imaging  (PMI)  developed  by  Artann  Labs,  is 
newly  FDA  approved  for  PCa  screening.  Unlike  conventional  B-mode  US,  PMI  allows  for  the 
measurement  of  gland  volume  and  mechanical  stress  patterns  on  the  gland  surface  through  the 
rectal  wall  with  pressure  sensor  arrays.  However,  PMI  lacks  the  high  spatial  resolution  of 
Magnetic  Resonance  Imaging  (MRI).  Multi-Parametric  MRI  (T2W,  T1W,  Diffusion,  and  DCE) 
has  recently  shown  great  promise  for  improving  screening  and  detection  of  PCa  as  it  provides 
significant  structural  and  functional  parameters  for  disease  characterization.  This  project 
developed  novel  computerized  fusion  methods  for  precise  registration  of  MRI  and  PMI  in  order 
to  enable  improved  PCa  detection  in  vivo  compared  to  using  either  PMI  or  MRI  alone.  This 
research  have  an  impact  on  development  of  reliable  and  practical  clinical  means  of  imaging  PCa 
for  improved  (1)  biopsy  planning  and  guidance,  (2)  grading  and  prognosis  formulation,  (3)  active 
surveillance,  (4)  treatment  planning,  (5)  disease  targeting,  and  (6)  treatment  monitoring. 
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Changes/Problems 

Nothing  to  report 
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Products 


Accepted  and  presented  abstracts: 

Mahdi  Orooji,  Mehdi  Alilou,  Rachel  Sparks,  Mirabela  Rusu,B  Nicolas  Bloch,  Ernest 
Feleppa,  Dean  Barratt,  Lee  Ponsky,  Anant  Madabhushi,  “A  Combination  of  Radiomic 
Features  from  MRI  and  Ultrasound  Appears  to  better  predict  presence  of  prostate  cancer: 
Validation  against  whole  mount  pathology”,  24th  International  Society  for  Magnetic 
Resonance  Medicine  (ISMRM),  7-13  May  2016,  Singapore. 

Orooji,  M.,  Rakshit,  S.,  Beig,  N.,  Velcheti,  V.,  Madabhushi,  A.,  "Computerized  textural 
analysis  of  lung  CT  enables  quantification  of  tumor  infiltrating  lymphocytes  in  NSCLC", 
American  Society  for  Clinical  Oncology  (ASCO)  Annual  Meeting,  Chicago,  IL,  2016 

Rakshit,  S.,  Orooji,  M.,  Beig,  N.,  Velcheti,  V.,  Madabhushi,  A.,  “Use  of  radiomic 
features  on  baseline  CT  scan  to  predict  clinical  benefit  for  pemetrexed  based 
chemotherapy  in  metastatic  lung  adenocarcinoma”,  American  Society  for  Clinical 
Oncology  (ASCO)  Annual  Meeting,  Chicago,  IL,  2016 

Mahdi  Orooji,  Mehdi  Alilou,  Niha  Beig,  Sagar  Rakshit,  Prabhakar  Rajiah,  Michael  Yang, 
Frank  Jacono,  Robert  Gilkeson,  Philip  Linden,  Vamsidhar  Velcheti,  Anant  Madabhushi, 
“A  combination  of  shape  and  texture  features  enables  discrimination  of  benign  fungal 
infection  from  non-small  cell  lung  adenocarcinoma  on  chest  CT”,  102nd  Scientific 
Assembly  and  Annual  Meeting  of  Radiological  Society  of  North  America  (RSNA),  Nov 
27-  Dec  3,  Chicago,  IL. 


Ongoing  Journal  Article: 

Mahdi  Orooji,  Mehdi  Alilou,  Lee  Ponsky,  Anant  Madabhushi,  “Spatial  Statistic  Aware 
Segmentation  Paradigm:  Application  on  the  Prostate  Segmentation  in  the  Transrectal 
Ultrasound  Images”,  is  going  to  submit  to  IEEE  Transaction  on  Medical  Imaging. 


US  Patent 

Madabhushi,  Anant  (Shaker  Heights,  OH,  US),  Rusu,  Mirabela  (Cleveland,  OH,  US), 
Orooji,  Mahdi  (Cleveland,  OH,  US),  Alilou,  Mehdi  (Cleveland,  OH,  US),  “Textural 
Analysis  of  Lung  Nodules”,  United  States  Patent  Application  20160155225. 
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Participants  &  Other  Collaborating  Organizations. 

•  What  individuals  have  worked  on  the  project? 

PI:  Mahdi  Orooji,  PhD:  No  change 
Mentor:  Anant  Madabhushi,  PhD:  No  change 
Co-mentor:  Lee  Ponsky,  MD:  No  change 
Collaborator:  Pingfu  Fu,  PhD:  No  change 
Collaborator:  Vikas  Gulani,  MD:  No  change 
Collaborator:  Raj  Paspulati,  MD:  No  Change 
Collaborator:  Armen  Sarvazian,  PhD:  No  change 
Collaborator:  Gregory  MacLennan,  MD:  No  change 

•  Has  there  been  a  change  in  the  active  other  support  of  the  PD/PI(s)  or  senior/key 
personnel  since  the  last  reporting  period? 

Nothing  to  Report 


•  What  other  organizations  were  involved  as  partners? 
Nothing  to  Report 
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Special  Reporting  Requirements 

Nothing  to  report 
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Appendices 

Part  1:  Accepted  Papers 


Abstract  #171681 

Computerized  textural  analysis  of  lung  CT  to  enable  quantification  of  tumor 
infiltrating  lymphocytes  in  NSCLC. 

Mahdi  Orooji,  Sagar  Rakshit,  Niha  Beig,  Anant  Madabhushi,  Vamsidhar  Veicheti; 
Biomedical  Engineering,  Case  Western  Reserve  University,  Cleveland,  OH;  Cleveland 
Clinic  Foundation,  Cleveland,  OH;  Case  Western  Reserve  University,  Cleveland,  OH; 
Cleveland  Clinic,  Cleveland,  OH 


Abstract  Text: 

Background:  Tumor  infiltrating  lymphocytes  (TILs)  are  a  part  of  the  dynamic  immune 
microenvironment.  Clinical  trials  with  immune  checkpoint  inhibitors  report  significant 
increase  in  TILs  in  responders  in  follow  up  tumor  biopsies.  Monitoring  TILs  on  treatment 
using  radiomic  features  extracted  from  routine  follow  up  computed  tomographic  (CT) 
images  can  be  a  non-invasive  surrogate  to  biopsy  to  detect  early  response.  We 
conducted  a  proof  of  concept  study  to  find  out  if  radiomic  features  extracted  from  CT 
images  can  identify  patients  with  high  and  low  TILs  in  non-small  cell  lung  cancer 
(NSCLC). 

Methods:  A  cohort  of  50  consecutive  patients  who  underwent  lobectomy  for  early  stage 
NSCLC  were  identified  and  TIL  were  characterized  using  routine  hematoxylin  and  eosin 
(H&Es)  slides.  TILs  were  quantified  on  a  previously  reported  scale  of  0  to  3  based  on 
intensity  of  TIL.  Of  the  50  cases  17  outliers  who  had  high  TILs  (3+)  or  low  TILs  (0)  were 
identified.  The  study  team  was  provided  with  CT  images  from  4  patients  with  ‘0’  TILs,  8 
tumors  with  ‘3+’  TILs  and  5  tumors  were  blinded.  Corresponding  pre-surgical  CT  scans 
were  annotated  on  slicer-3D  software.  A  total  of  669  radiomic  (textural  and  shape) 
features  of  the  lung  nodule  were  investigated.  These  features  were  evaluated  and 
ranked  in  their  ability  to  discriminate  TIL  extent  using  a  linear  discriminant  classifier, 
both  in  terms  of  univariate  and  multivariate  analysis. 

Results:  Sphericity,  a  shape  based  feature  was  the  most  discriminating 
feature.  Standard  deviation  and  Laws  were  ranked  as  the  most  promising  texture  based 
features.  Of  the  5  blinded  tumors,  4  were  classified  correctly  leaving  only  one 
misclassified  case. 

Conclusions:  Computerized  textural  analysis  using  shape  and  texture  features 
extracted  from  the  lung  nodule  on  CT  images  could  be  used  to  identify  tumors  with  high 
TILs.  Further  validation  of  these  findings  in  larger  independent  cohorts  is  required. 
These  novel  imaging  based  biomarkers  could  be  a  useful  diagnostic  tool  for  predicting 
response  and  monitoring  patients  on  immunotherapy. 
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Abstract  #171139 

Evaluation  of  radiomic  features  on  baseline  CT  scan  to  predict  clinical  benefit  for 
pemetrexed  based  chemotherapy  in  metastatic  lung  adenocarcinoma. 

Sagar  Rakshit,  Mahdi  Orooji,  Niha  Beig,  Mehdi  Alilou,  Nathan  A.  Pennell,  James 
Stevenson,  Marc  A.  Shapiro,  Anant  Madabhushi,  Vamsidhar  Velcheti;  Cleveland  Clinic 
Foundation,  Cleveland,  OH;  Biomedical  Engineering,  Case  Western  Reserve  University, 
Cleveland,  OH;  Cleveland  Clinic,  Cleveland,  OH;  Case  Western  Reserve  University, 
Cleveland,  OH 

Abstract  Text: 

Background:  Many  patients  receiving  standard  of  care  pemetrexed  based  platinum 
doublet  followed  by  maintenance  pemetrexed  for  lung  adenocarcinoma  do  not  receive 
clinical  benefit.  Currently  there  are  no  clinically  validated  biomarkers  to  identify  patients 
who  benefit  from  these  treatments.  We  conducted  a  retrospective  proof-of  concept 
study  to  identify  predictive  computer  extracted  image  features  from  pre-treatment 
computed  tomographic  (CT)  scans. 

Methods:  Pre-chemotherapy  CT  scans  were  obtained  for  105  lung  adenocarcinoma 
patients  treated  with  pemetrexed  based  chemotherapy  at  the  Cleveland  Clinic  from 
2004-2010.  Clinical  benefit  was  defined  as  patients  with  an  objective  response  or  more 
than  12  cycles  of  pemetrexed  therapy.  We  identified  and  annotated  CT  images  in  2 
groups-  46  with  clinical  benefit  and  59  without  clinical  benefit.  After  adjusting  for  image 
quality  and  CT  filters,  32  and  33  patients  remained  for  final  analysis  in  the  2  groups 
respectively.  A  total  of  1108  radiomic  features  including  both  textural  and  shape 
features  of  the  lung  nodule  as  well  as  the  peritumoral  region  were  investigated.  The 
features  were  evaluated  and  ranked  in  their  ability  to  discriminate  between  the  2  groups 
in  conjunction  with  a  linear  discriminant  classifier,  both  in  terms  of  univariate  and 
multivariate  analysis. 

Results:  Two  of  the  top  3  ranked  features  were  from  within  the  nodule  and  the  third 
was  from  within  the  peritumoral  area.  Mean  of  Sum  Average,  a  co-occurrence  based 
texture  measure  within  the  nodule  was  the  most  discriminating  feature.  Combination  of 
features  within  and  around  the  nodule  yielded  even  higher  AUC  values  (See  Table  for 
combination  of  best  features). 

Conclusions:  Texture  and  shape  features  extracted  from  within  and  around  the  lung 
nodule  on  CT  images  could  identify  patients  who  could  potentially  benefit  from 
pemetrexed  based  chemotherapy.  Further  validation  in  a  larger  retrospective,  multi- 
institutional  cohort  is  needed. 


Feature  Vector  Area  Under  ROC  Curve  ±  Standard  Deviation 

Mean  of  Intratumoral  Sum  Average(l)  69.5%  ±  3.0% 

1+Minimum  of  Intratumoral  Law  L5xE5(2)  75.2%  ±  3.2% 

1+2+Mean  of  Peritumoral  Law  S5xW5(3)  77.6%  ±  1.9% 
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A  Combination  of  Radiomic  Features  from  MRI  and  Ultrasound  Appears  to 
better  predict  presence  of  prostate  cancer:  Validation  against  whole  mount 
pathology 

Authors:  Mahdi  Orooji1,  Mehdi  Aliiou1 ,  Rachel  Sparks2,  Mirabela  Rusu\B  Nicolas  Bloch3, 
Ernest  Feleppa4,  Dean  Barratt2,  Lee  Ponsky5,  Anant  Madabhushi1 

1  Center  for  Computational  Imaging  and  Personalized  Diagnostics,  Case  Western  Reserve 
University,  Cleveland,  OH,  USA 

2Centre  for  Medical  Image  Computing,  University  College  of  Uondon 
3Boston  University  School  of  Medicine,  Boston,  Massachusetts 
4Lizzi  Center  for  Biomedical  Engineering,  Riverside  Research 

5University  Hospital  Case  Medical  Center,  Case  Western  Reserve  University,  Cleveland,  OH, 
USA 


Synopsis: 

To  evaluate  whether  the  combination  of  computer  extracted  or  radiomic  image 
parameters  from  two  complementary  modalities,  MRI-Transrectal  Ultrasound  (TRUS) 
can  enable  better  prediction  of  presence  of  prostate  cancer  compared  to  either  modality 
individually.  In  order  to  evaluate  the  ability  of  the  radiomic  features  from  MRI  and 
ultrasound  and  the  combination  of  MRI  and  ultrasound  radiomic  features  in  predicting 
the  presence  of  prostate  cancer  we  considered  3  patients  who  underwent  MRI, 
transrectal  ultrasound  prior  to  radical  prostatectomy.  Deformable  co-registration 
methods  were  used  for  spatially  aligning  the  pre-operative  in  vivo  MRI  and  ultrasound 
with  the  ex  vivo  whole  mount  radical  prostatectomy  specimens  to  establish  the  “ground 
truth”  for  cancer  extent  on  the  imaging.  A  combination  of  texture  features  from  US  and 
MRI  yielded  the  best  separability  between  cancer  and  non-cancer  regions  with  an  Area 
under  the  operating  characteristic  curve  of  0.88. 

PURPOSE: 

Recently  there  has  been  a  great  deal  of  interest  in  developing  computer  aided  diagnosis 
systems  for  identifying  prostate  cancer  presence  in  vivo  on  MRI  and  ultrasound 
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separately1-3,  no  work  we  are  aware  of  has  attempted  to  address  the  issue  of  fusing 
computer  derived  features  from  MRI  and  ultrasound  to  create  the  best  possible 
predictor  of  cancer  in  vivo.  In  this  work  we  attempt  a  systematic  and  quantitative 
evaluation  of  the  discriminability  of  computer  extracted  MRI  and  ultrasound  features  in 
terms  of  cancer  detection  in  patients  undergoing  radical  prostatectomy. 

METHODS: 

Our  study  design  comprised  12  2D  planar  images  obtained  from  the  MRI  and  US  scans 
of  3  patients,  all  of  whom  had  biopsy  confirmed  prostate  cancer  and  scheduled  for  a 
radical  prostatectomy.  A  3D  B-mode  ultrasound  scan  was  performed  followed  by  a  3 
Tesla  MRI  prior  to  surgery.  Following  surgery  and  histologic  sectioning  of  the  gland  via 
a  microtome,  the  H&E  stained  whole  mount  histologic  (WMH)  sections  were  digitized 
via  a  whole  slide  scanner  and  the  regions  of  cancer  annotated  by  an  expert  pathologist. 
Deformable  co-registration  methods  were  used  to  spatially  align  the  in  vivo  MRI,  TRUS, 
and  ex  vivo  histology.  In  particular,  we  used  fully  automatic  Multiattribute  probabilistic 
prostate  elastic  registration  (MAPPER)  approach  to  fusion  of  ultrasound  and  MRI4.  We 
also  manually  delineated  corresponding  landmarks  between  MRI  and  WMH  for 
deformable  co-registration  of  WMH  to  MRI.  A  total  of  129  computer  extracted  image 
features  including  Haralick,  Gabor,  Law,  LBP,  Laplacian  features  were  extracted  from 
both  the  prostate  MRI  and  TRUS.  Each  of  the  computer  extracted  MRI  and  ultrasound 
features  were  then  ranked  via  the  Fisher  criteria  to  identify  the  features  that  best 
identified  the  region  of  cancer.  Figure  1  illustrates  the  MRI-TRUS-WMH  registration  and 
mapping  of  the  cancer  extent  on  MRI  and  TRUS. 

RESULTS  AND  DISCUSSION 

The  top  3  features  for  each  modality  and  corresponding  Fisher  criteria  values  are  shown 
in  Table. 1.  The  classification  is  per  region  of  interest  (ROI),  i.e.  the  texture  features  for 
the  cancerous  part  of  the  prostate  is  compared  to  the  texture  features  of  the 
noncancerous  confounding  regions.  Top  three  texture  features,  contrast  variance, 
contrast  entropy,  and  contrast  inverse  moment  were  selected  by  the  theoretical  linear 
discriminant  analysis  (LDA)  classifier  for  MRI  and  yielded  an  area 
under  the  receiver  operating  characteristic  curve  (AUC)  of  0.83,  0.77  and  0.70 
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for  identifying  cancerous  ROIs  in  MRI.  By  comparison,  top  three  most  predictive 
features  identified  for  TRUS  were  contrast  inverse  moment,  contrast  variance,  and 
contrast  entropy.  These  features  yielded  an  AUC  of  0.75,  0.69,  and  0.66,  respectively. 
By  combining  the  top  two  texture  features  on  MRI  and  the  most  informative  texture 
feature  on  TRUS,  the  LDA  based  predictor  yielded  an  AUC  of  0.88  in  predicting 
presence  of  prostate  cancer.  Figure  2  illustrates  the  scatter  plot  of  the  prostate  cancer 
versus  the  non-cancer  cases  in  three  dimensional  most  informative  texture  feature 
space. 

CONCLUSION: 

We  presented  a  framework  to  rank  the  performance  of  computer  extracted  MRI  and 
ultrasound  features  in  terms  of  their  ability  to  identify  prostate  cancer.  Our  results  in  a 
small  cohort  suggests  that  we  may  be  able  to  combine  the  MRI  and  ultrasound  radiomic 
features  to  create  a  better  classifier  for  prostate  cancer  detection  compared  to  MRI  or 
ultrasound  alone. 
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Figure  1:  Registration  of  MRI,  TRUS  and  WMH:  Two  2D  planar  images  of  (a),(g)  WMH  and  (b),(h) 
corresponding  MRI.  (c),(i)  WMH  and  MRI  checkerboard  overlays  showing  alignment  between  the  two 
modalities.  (d),(j)  MRI  with  cancer  annotation  obtained  from  WMH  (green).  (e),(k)  TRUS  with  cancer 
annotation  obtained  from  WMH  (green).  (f),(l)  Fused  MRI- TRUS  images  shown  as  checkerboards  with 
cancer  annotation  obtained  from  WMH  (green). 
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Figure  2:  Scatter  plot  of  three  most  discriminative  texture  features 


Table  1:  Top  3  features  of  MRI  and  TRUS,  and  their  AUC  values 


AUC 

0.83 

Entropy 

0.77 

Features  for  TRUS 

Contrast.  Inverse  Moment 

Contrast 

Variance 

AUC 

0.75 

0.69 

Inverse  Moment 
0.70 
Contrast 
Entropy 
0.66 
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Abstract — Prostate  needle  biopsy  guided  by  transrectal  ultra¬ 
sound  (TRUS)  is  the  current  gold  standard  for  prostate  cancer 
(CaP)  detection.  An  accurate  detection  of  the  prostate  volume 
and  boundary  is  influential  on  diagnosis,  treatment,  and  follow 
up  of  CaP.  Within  this  framework,  prostate  segmentation  on 
TRUS  imagery  is  performed  via  introduced  spatial  statistic 
aware  segmentation  paradigm.  The  spatial  prior  probability  is 
calculated  in  the  training  phase,  and  is  used  to  estimate  the 
texture  feature  parameters  corresponding  to  the  prostate  and 
the  background.  Estimated  parameters  is  employed  to  represent 
an  alternative  probabilistic  presentation  of  the  prostate  in  TRUS. 
Results  show  the  3D  prostate  capsule  is  more  pronounced  in  the 
new  representation  which  ultimately  results  in  more  accurate 
segmentation.  Modified  active  shape  model  (ASM)  is  introduced 
and  applied  in  the  new  3D  TURS  representation  for  prostate 
segmentation. 

Index  Terms — Prostate  Segmentation,  Texture  Feature 
Heatmap,  Parameter  Estimation,  Expectation-Maximization, 
Maximum  Likelihood  Estimation,  Active  Shape  Model 

I.  Introduction 

Prostate  cancer  (CaP)  is  the  second-leading  cause  of  cancer 
deaths  in  men  and  leading  cause  of  cancer  deaths  men  over 
age  50  in  the  United  States.  It  is  estimated  450,000  new  CaP 
case  swill  be  detected  by  2015.  Predictions  also  suggest  one 
fourth  of  new  CaP  cases  will  be  in  men  less  than  65  years 
of  age  which  emphasizes  the  necessity  of  early  detection  and 
localization  of  CaP.  Prostate  needle  biopsy  guided  by  transrec¬ 
tal  ultrasound  (TRUS)  is  the  gold  standard  for  CaP  diagnosis. 
Determining  the  prostate  volume  and  identifying  the  prostate 
capsule  plays  a  crucial  role  in  needle  targeting.  Due  to  the 
importance  in  delineating  the  prostate  on  TRUS,  semi  and  fully 
automated  prostate  segmentation  methods  for  TRUS  imagery 
have  been  developed.  Despite  recent  advances  in  transducer 
design,  resulting  in  improved  spatial  and  temporal  resolution, 
TRUS  image  segmentation  is  still  heavily  influenced  by  the 
quality  of  data  [1],  [2],  Additionally,  TRUS  segmentation  is 
vulnerable  to  variety  of  artifacts,  such  as,  different  levels 
of  signal  attenuation,  shadowing  artifacts,  and  speckle.  Low 
contrast  between  areas  of  interest  is  another  difficult  problem 
in  prostate  segmentation  on  TRUS. 

Previous  prostate  segmentation  algorithms  for  TRUS  im¬ 
age  have  focused  on  incorporating  prior  knowledge  such  as 
prostate  shape  [3]  or  prior  spatial  probability  [4],  Recently,  2D 
or  3D  boundary  extraction  methods  have  been  developed  based 


on  probabilistic  data  association  filters  [2).  However,  these 
approaches  assume  prior  information  for  the  prostate  shape 
(such  as  concavity)  and/or  need  manually  placed  initial  seed 
locations  inside  the  prostate  [5],  Recently,  active  contour-based 
approaches  have  been  used  for  prostate  segmentation.  These 
methods  assumed  prior  spatial  properties  [6],  or  needs  some 
initial  points  on  [7]  or  near  [8]  the  prostate  boundary.  These 
methods  have  also  been  employed  to  fuse  multi-modal  images 
of  the  prostate  [9].  Our  method  is  a  fully  automated  semi- 
supervised  segmentation  algorithm  that  uses  only  the  spatial 
prior  probability  for  parameter  estimation.  Feature  parameter 
estimation  considers  all  voxels  in  an  image;  however,  segmen¬ 
tation  is  performed  on  each  voxel  independently.  Considering 
joint  probability  of  extracted  features  and  other  criteria  such 
as  restricted  maximum  distance  of  voxels  in  the  segments  or 
morphology  of  the  segments  [10],  the  presented  method  can 
be  generalized  to  exploit  prior  particular  available  knowledge. 

In  this  work,  we  formulize  a  spatial  statistic  aware  seg¬ 
mentation  paradigm,  and  applied  it  for  3D  segmentation  of 
the  prostate  in  the  TRUS.  Using  introduced  paradigm,  a 
distribution  parameters  of  129  texture  features  in  TRUS  is 
estimated  and  the  prostate/background  probability  heatmap 
is  generated  via  using  estimated  distribution.  A  more  infor¬ 
mative  alternative  representation  of  the  prostate  in  TRUS  is 
introduced,  and  modified  active  shape  model  is  applied  on 
it  for  automatic  annotation  of  the  prostate  boarder.  The  rest 
of  the  article  is  organized  as  follows.  After  the  introduction 
in  the  first  section,  to  increase  the  readability,  in  the  second 
section  the  assumptions  are  described,  the  parameters  are 
defined,  and  the  problem  of  matter  is  formulized.  The  spatially 
aware  distribution  parameter  estimation  is  elevated  in  the 
third  section.  Estimated  distribution  is  employed  in  the  forth 
section  to  introduce  a  new  TRUS  HeatMap  Representation 
(THR).  In  section  five,  a  modified  adaptive  active  shape  model 
is  introduced  and  applied  on  THR.  The  performnce  of  the 
method  is  evaluated  in  section  six  and  the  paper  is  concluded 
in  section  seven. 

II.  Assumptions  and  Problem  Formulation 

As  a  problem  statement,  the  goal  is  to  segment  the  prostate 
in  a  TRUS  image  slide  denoted  by  I.  Note  that,  I  is  a 
2D  axial  section  of  a  3D  prostate  gland.  The  3D  capsule 
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would  be  reconstructed  by  stacking  the  estimated  prostate 
capsule.  To  increase  the  readability  of  the  manuscript,  we 
summarize  the  notation  we  used  in  the  following.  Here  and 
subsequently,  a,  Ayxx.  A*,  and  |  A|  imply  a  is  a  vector, 
A  is  a  matrix  with  Y  rows  and  X  columns,  a  transpose  of 
matrix  A ,  and  determinant  of  matrix  A,  respectively.  Suppose 
N  denotes  the  total  number  of  voxels  of  I.  By  extracting  L 
features  from  the  original  intensity  image,  we  can  reconstruct 
an  observation  matrix,  Dyxt  =  [dj, d2,---  . d,y]t  where  the 
vector  dn  -  {dn>i,dn<2,- ■  ■  ,dn<L}  for  n  e  {1,2,---  ,N} 
is  a  feature  vector  corresponding  to  each  voxel  of  I.  To 
keep  the  generality  of  approach,  I  is  desired  to  be  seg¬ 
mented  into  K  distinct  sets  of  voxels,  Si,  ■  ■  ■  ,  Sk  such 
that  the  union  of  all  segments  cover  the  entire  image,  i.e., 
U*Li  Sfc  =  J  and  njtLi  S/r  =  0.  The  set  of  parameters 
of  each  segment  that  needs  to  be  estimated  is  denoted  by 
0  =  {Ad, A}  in  which  Mkxl  =  for 

pk  =  {pk  '  •  •  ,  gk.L  }  is  the  mean  of  the  fcth  segment. 

A2  =  .S/cIxl]  is  the  set  of  covariance 

matrices  of  each  segments  where  Ei-2  =  [c*2j].  Table  II  lists 
the  notations  used  throughout  this  paper. 

A.  Spatial  Prior  Probability 

“.v x  k  =  [Cn.fc].  referred  to  as  the  Spatial  Prior  Probability, 
is  the  likelihood  of  the  nth  voxel  belonging  to  fcth  segment, 
just  based  on  its  spatial  information  relative  to  the  center  of  the 
ultrasound  probe  in  an  axial  TRUS  image.  So,  ^2k=1  £n,fc  =  1, 
for  any  given  n.  E  is  calculated  from  a  set  of  J  training  studies, 
where  for  each  study  the  prostate  has  been  segmented  by  an 
expert.  The  origin  for  each  study  is  set  as  the  center  of  the 
TRUS  probe,  so  that  the  location  of  voxels  have  a  consistent 
position  relative  to  the  TRUS  probe  across  all  studies.  =  i 
for  voxel  n  of  jth  training  image  belonging  to  class  k  where 
k  =  1  represent  prostate  and  k  =  0  represent  background. 
So,  each  element  of  the  spatial  prior  probability  matrix  is 
calculated  by  averaging  over  all  training  images  as: 

1  J 

Uk=jJ29n]  (1) 

3= 1 

Hence  £n,i  is  the  frequency  of  the  nth  voxel  being  located  in 
the  prostate  across  J  training  images. 

The  prostate  is  divided  into  three  regions,  the  apex,  the 
midgland  and  the  base  such  that  they  have  equal  length  in 
the  sagittal  axis.  Two  disjoint  spatial  prior  probability  was 
calculated  corresponding  to  the  region  of  interest,  one  for 
midgland  and  another  one  for  the  union  of  the  apex  and  the 
base. 


B.  Feature  Set  Probability 

Pd(dn;Q)  is  the  probability  distribution  function  (PDF)  of 
a  set  of  features  associated  with  voxel  n,  given  the  parameters 
(which  are  unknown).  Without  loss  of  generality,  we  assume 
Pd(d„;Q)  is  modeled  as  a  multivariate  Gaussian  distribution 
[4],  Note  that  even  the  intensity  of  the  ultrasound  image 
is  Rayleigh  distributed  [11],  [12];  however,  considering  the 
central  limit  theorem  under  weak  dependence  condition,  the 


Gaussian  distribution  of  the  extracted  texture  features  is  a  valid 
assumption  [13]. 

Matrix  of  Segments:  To  make  the  feature  set  parameters 
estimation  problem  tractable,  we  define  an  auxiliary  (latent) 
variable  Z^xK  =  a  Matrix  of  Segments,  such  that  any 

arbitrary  nth  row  of  Z  has  only  one  element  of  1  at  7.-th  column 
which  implies  nth  voxel  of  I  belongs  to  Sk ,  and  the  rest  of 
the  elements  of  the  nth  row  are  zero. 

Parameter  estimation  is  the  solution  of  the  following  max¬ 
imum  likelihood  equation: 

0  =  argmaxP(73|0,H) 
e 

=  arg  max  Y'  log  (P(D\Z,Q.E)P(Z\Q,E)^.  (2) 

The  conditional  distribution  of  the  observation  matrix  given  0 
and  Z  is  given  by: 


P(D\Z,e,E)=  f]P(dn|Z,0) 


N  K 

MB  l(2-)l/2i^i 


exp 


({dn-gk)'Y.-\dn-gk)) 


(3) 


The  matrix  of  segments  is  independent  of  the  distribution 
parameters  and  its  probability  value  is  a  function  of  spatial 
prior  probability  matrix  as  follows: 


P(Z|0,H)  =  P(Z|E)  =  n  n  (4) 

n=l k= 1 

One  should  note  that  for  other  feature  distributions  rather  than 
Gaussian  (such  as  the  distribution  of  the  intensity  in  TRUS 
images  which  is  Rayleigh  distribution  [11],  [12]),  (3)  needs  to 
be  substituted  by  a  proper  corresponding  distribution  function. 


III.  Spatial  Aware  Distribution  Parameter 
Estimation 

It  is  assumed  that  the  TRUS  image  is  consist  of  two 
segments,  the  prostate  and  the  background.  Constrained  to  the 
calculated  prior  spatial  probability  in  (1),  we  employed  an 
iterative  expectation-maximization  algorithm  to  estimate  the 
parameters  associated  with  the  prostate  and  the  background 
in  the  TRUS  for  each  individual  texture  feature.  In  the  other 
words,  for  a  given  observation  matrix  D,  we  estimate  the  ma¬ 
trix  of  parameters,  {M,  A2}.  Using  the  estimated  parameters, 
we  introduced  a  new  representation  of  the  TRUS  in  section 
IV. 

It  is  guaranteed  that  the  expectation-maximization  algorithm 
converges  to  the  local  maximum  likelihood,  however,  the 
convergence  of  it  to  the  global  maximum  and  the  legitimacy 
of  the  results  is  very  reliant  on  the  initialization  [14]. 

To  improve  the  parameter  estimation,  we  introduced  param¬ 
eter  estimation  method,  namely  Decoupled  Maximum  Like¬ 
lihood  (DML),  that  employed  the  spatial  prior  probability 
for  a  sub-optimum  maximum  likelihood  estimation.  The  sub¬ 
optimum  estimated  parameters  are  used  an  initial  estimation 
of  the  parameters  in  the  expectation-maximization  algorithm. 
Our  assessments  shows  the  introduced  method  converges  to 
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TABLE  I 

Description  of  notation  used  throughout  this  paper. 


Notation 

Description 

Notation 

Description 

i 

TRUS  image  scene. 

0  =  {.M.A} 

Set  of  actual  parameters. 

N 

Total  number  of  pixels. 

e  =  {At,  A} 

Set  of  estimated  parameters. 

L 

Total  number  of  extracted  features. 

Ok  =  [mjmIixl 

Vector  of  actual  means  of  A.-th  segment. 

K 

Assumed  number  of  segments. 

Uk  =  [Afc,l]lxl. 

Vector  of  estimated  means  of  kth  segment. 

Dnxl 

Matrix  of  observations. 

££  =  [<  Jlxl 

Set  of  actual  covariance  matrix  of  kth  segment. 

d„ 

Features  vector  of  nth  voxel. 

Z'i  =  [*i IJlxl 

Set  of  estimated  covariance  matrix  of  kth  seg¬ 
ment.. 

=■  =  [?n,fc]lVXK 

Spatial  prior  probability  matrix. 

Ififi) 

Fisher  Information  of  parameter  0. 

the  global  maximum  with  less  number  of  the  iterations  and 
does  not  trap  on  the  local  maximum. 

A.  Decoupled  Maximum  Likelihood  (DML)  Approach  for  Ini¬ 
tial  Estimation  of  the  Parameters 

By  substituting  corresponding  values  from  (3)  and  (4)  into 
(2),  one  can  note  that  the  maximum  likelihood  problem  does 
not  have  an  analytical  closed  form  solution.  When  the  analyt¬ 
ical  solution  does  not  exist,  generalized  maximum  likelihood 
ratio  test  (GLRT)  is  proposed  [15].  GLRT  is  not  an  optimum 
solution,  because  instead  of  maximizing  the  likelihood  ratio, 
independently  treats  the  nominator  and  the  denominators  of 
the  likelihood  ratio  and  maximize  them  separately  [16].  So,  it 
results  in  the  sub-optimum  answer.  Using  GLRT  algorithm,  we 
introduced  the  decoupled  maximum  likelihood  to  estimate  the 
parameters  of  the  nominator  (corresponding  to  the  prostate) 
and  the  denominator  (corresponding  to  the  background)  sepa¬ 
rately.  The  algorithm  is  as  follows. 

The  parameters  of  fcth  segment  is  denoted  by  0k  —  {/4.  ££} 
for  k  =  1  and  2  corresponding  to  the  background  and 
the  prostate,  respectively.  t)j:DMl1  are  estimated  parameters  of 
Lth  segment  obtained  by  and  decoupled  maximum  likelihood 
approach.  So, 


B.  Parameters  Estimation  Per  Individual  Texture  Feature 

The  solution  of  the  Maximum  Likelihood  (ML)  Ratio  Test 
for  each  texture  feature  is  the  optimum  unsupervised  parameter 
estimation  approach  [15],  [17].  However,  ML  is  too  compli¬ 
cated  to  be  derived  analytically.  Hence,  we  employ  an  iterative 
algorithm,  inspired  by  the  Expectation-Maximization  (EM) 
mixture  model  parameter  estimation  approach,  to  estimate 
values  of  0.  EM  is  a  powerful  iterative  algorithm  to  estimate 
the  parameters  of  the  mixture  models  when  the  associated  log- 
likelihood  maximization  problem  is  too  complicated  to  solve 
analytically.  Using  the  same  approach,  we  modified  the  two 
steps  of  Expectation  and  Maximization  as  follows: 

Expectation  Step:;  Current  and  revised  estimation  of  0  are 
denoted  by  ©0LD  and  0NEW,  respectively.  The  conditional  ex¬ 
pectation  of  log  P(D,  Z |0)  given  D  and  the  current  estimation 
of  0  is  given  by, 

Q(0;  ©0LD)  =  Ez  [log  P(D,  Z\&)]  (9) 

=  [*„,*|£>,e0LD]  x 

n=l k= 1 

(log£„,fc  -  ^  log  27T  -  log  IS  I  -I-  {dn  -  pkyz,^2(d„  -  £*)) 


<?[DML)  =  argmax  P(D\0k,E),  for  1  <  k  <  K  (5) 

where  i?2[.|a:]  is  the  conditional  expectation  given  x,  respect 
From  (5),  it  is  implied  that  the  parameter  estimation  problem  to  z.  After  some  manipulation  one  can  show  that: 
for  each  segm 
(5)  we  have, 

P(D\ek,E) 

N 

exp  | 


2  L(2rr)L|S^| 


7 n,k  —  EZ 

D,  0OLD] 

(10) 

(6) 

pk)'Xk2(dn-pk))j  ^ 

£,n,k  P 

(<?n  2r.,fc>©°LD) 

£fc=I  fn,* 

:P(d„  2„,fc,©0LD) 

After  some  manipulation  one  can  show  that  the  solution  of 
DML  parameter  estimation  from  (6)  is  given  by: 

j^p(D\ek,  s)  =  o  =*  /4=  Ukdn  (7) 

E„=i  Uk 

^  n(  TA\n  _  n  _ v2  _  5Zn=l  £>n,k{dn  t^k) 

L,- 

k  /  jn  —  1  S7l,K 

(8) 


Maximization  Step:;  It  has  been  shown  in  [18]  that  the  log- 
likelihood  function  is  the  monotone  increasing  function  of  the 
number  of  EM  iteration  steps.  So,  regardless  of  the  initial  value 
of  0,  EM  converges.  ©NEW,  the  new  values  of  parameters,  are 
the  solution  of: 

0NEW  =  arg  max  Q  (0, 0OLD)  (1 1) 

e 


The  estimated  mean  and  variance  from  (7)  and  (8)  are 
employed  as  an  initial  mean  and  variance  in  the  next  step. 


By  calculating  the  partial  derivative  of  (9)  and  (10)  respect  to 
pt,  and  £jt,  one  can  show  that 
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(14) 


~  _  En=l  7 n,kdn,l 

A*M  —  N  i 

E  n-1  Tn.fc 

i.2  _  En= l'yn,k(dn,l  /^fc.i)2 

ak,l  ~  „.y 

2_/n=l  Tn.fc 


(12) 

(13) 


IV.  TRSU  HeatMap  Representation  (THR) 

Prostate  segmentation  in  TRUS  is  a  challenging  task  mainly 
due  to  the  high  inherent  noise  of  the  ultrasound  images  [19]. 
On  the  other  side,  the  proficiency  of  the  texture  features  in 
the  prostate  segmentation  is  already  investigated  [20],  We 
employed  the  power  of  the  texture  features  in  presenting 
the  prostate  in  the  ultrasound  images  and  also  the  spatial 
statistics  information  to  introduce  an  alternative  representation 
of  the  TRUS  image,  in  which  the  prostate  capsule  is  more 
pronounced  in  compare  to  the  original  TRUS  image.  We 
named  it  the  TRUS-HeatMap  Representation  (THR)  . 

To  generate  the  THM  representation,  we  need  to  generate 
the  heatmap  probability  of  the  foreground  and  the  background 
per  feature.  To  do  so,  the  first  step  is  calculating  the  spatial 
prior  probability  by  substituting  the  prostate  capsule  annota¬ 
tions  in  (1).  Because  the  size  of  the  prostate  capsule  in  the 
apex  and  the  base  are  close  and  it  is  relatively  smaller  than 
the  size  of  the  prostate  in  the  midgland  [21],  so,  we  generated 
two  disjoint  spatial  prior  probability  model  for  two  sets  of  the 
apex/based  and  midgland. 

In  the  second  step  texture  features  were  extracted  for  a  given 
TRUS  image.  In  this  articled  we  employed  total  of  129  texture 
features  which  described  in  Table  II.  Considering  the  spatial 
prior  probability,  in  the  third  step  we  employed  the  Spatiall 
Aware  Distribution  Parameter  Estimation  Method  introduced 
in  section  III  to  answer  which  segments  in  conjunction  with 
which  set  of  parameters  are  going  to  maximize  the  likelihood 
of  observed  extracted  texture  features  of  a  TRUS  image.  To 
improve  the  proficiency  of  EM-based  parameter  estimation 
method,  as  it  was  described  in  the  subsection  III-A,  we 
included  the  Decoupled  Maximum  Likelihood  approach  for 
initial  estimation  of  the  parameters  in  the  third  step. 

And  finally  in  the  fourth  step,  the  estimated  distribution  pa¬ 
rameters  corresponding  to  the  foreground  and  the  background 
for  each  feature  is  used  to  generate  the  foreground  and  the 
background  probability  heatmap.  Figure  1  demonstrates  the 
heatmap  generation  schematic  for  a  texture  feature.  Figure  2 
demonstrates  the  heatmap  of  the  prostate  and  the  background 
for  Haralick  features  [22]. 


A.  Adaptive  Combination  of  the  Texture  Feature ’s  Heatmaps 
to  Generate  THR 

After  calculating  the  probability  heatmap  of  the  foreground 
and  the  background  for  each  feature,  we  need  to  find  which 
feature  in  the  foreground  and  which  feature  in  the  background 
has  the  highest  capability  of  the  prostate  segmentation.  To 
do  so,  we  need  to  define  a  proficiency  metric  for  foreground 
heatmaps  and  the  background  heatmaps. 


i  A 

<PPr  = 
<pBk  — 


Evn  ~n.lfpr(dn) 

Evn  zn,  1 

Ev„  *n.O  (1  ~  Fr(dn)) 


(15) 


By  combining  the  estimated  probability  of  the  prostate  and 
the  estimated  probability  of  the  background,  we  generated 
the  prostate  probability  heat-map.  Defined  proficiency  metric 
is  employed  to  find  the  most  proficient  texture  features  for 
discriminating  the  prostate  from  the  background.  So,  we  added 
the  proficiency  weighted  heatmaps  to  obtain  the  final  prostate 
heatmap.  Finally,  AAM  is  applied  on  the  prostate  probability 
heatmap  from  the  previous  step  for  prostate  segmentation  in 
TRUS.  Figure  3  demonstrates  the  prostate  heatmap. 


B.  Calculating  the  Heat-map 

Estimated  features  parameters,  9,  induce  maximum  likeli¬ 
hood  image  segmentation  regardless  of  the  parameter  estima¬ 
tion  approach  (DML  or  ML).  The  prostate  heat-map  is  defined 
as  the  conditional  probability  (given  estimated  parameters)  that 
a  voxel  n  belongs  to  the  prostate  which  would  be: 

P(z  „,i  =  1|9)  (16) 

=  jfn.l-f*  (dn  |  /tl,  +  £n, 0  ~  P  (dn  I  Ao,  ] 

where  'jl  is  a  normalization  factor  to  guarantee  that  the  heat¬ 
map  values  are  in  the  range  of  0  to  1 . 

V.  Adaptive  Active  Shape  Model  on  TRUS 
HeatMap 

VI.  Experimental  Results 

A.  Dataset 

The  presented  EM-based  segmentation  method  was  applied 
on  3D  Transrectal  Ultrasound  (TRUS)  images  for  six  patients. 
TRUS  images  was  acquired  using  a  bi-planer  side-firing  tran¬ 
srectal  probe.  A  prostate  boundary  on  TRUS  was  manually 
delineated  by  an  expert  radiologist.  To  correct  for  the  possible 
TRUS  attenuation  artifacts  (due  to  the  fact  that  pixels  closer 
to  the  probe  appear  brighter),  we  employed  the  method  in  [4], 
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TABLE  II 

Description  of  the  texture  features  used  for  prostate  segmentation  in  TRUS. 


Feature  category 

Descriptor 

Intuitive  Description 

Haralick  features 
(Repeated  occurrence  of 
grey  level  configuration 
in  the  texture  represented 
via  the  grey-level  co- 
occurrence  matrix 
(GLCM),  which  varies 
rapidly  with  distance  in 
fine  textures  and  slowly 
in  large  textures) 

Inverse  Difference 
Moment  (IDM) 

IDM  is  a  reflection  of  the  presence  or  absence  of  uniformity,  and  hence  is  a 
measure  of  local  regions  of  homogeneity 

High  IDM:  Higher  presence  of  locally  uniform  windows  in  GLCM 

Low  IDM:  Higher  presence  of  locally  heterogeneous  windows  in  GLCM 

Correlation 

Quantifies  the  linear  patterns  in  an  image  based  on  the  distance  parameter. 

Sum  Entropy 

Measure  of  GLCM  relationship  to  distribution  of  intensity  with  respect  to  entropy. 
Entropy  is  the  measure  of  disorder. 

Sum  Variance 

Measure  of  GLCM  relationship  to  distribution  of  intensity  with  respect  to  variance 
High  sum  variance:  greater  standard  deviation  of  sum  average 

Low  sum  variance:  low  standard  deviation  of  sum  average 

Laws  features 

E5,  L5,  S5 

Combining  E-  Edges.  L-  Level  and  S-  Spots  in  both  X  any  Y  directions 

Laplacian  pyramids 

Multi-resolution  filters  capture  edges  at  different  levels 

Gray  level  features 

The  basic,  intensity  based  features  including  mean,  median,  range  and  standard 

deviation. 

Gabor  Features 

Oriented  textures  via  changes  in  direction  and  scale;  capture  microarchitecturcs 

Gradient  Features 

Represent  the  directional  change  in  the  intensity  values  of  pixels  in  the  ROI 

Local  Binary  Pattern 

Thrcsolding  the  window  with  the  center  pixel  value. 

Training  Annotations 
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Fig.  1 .  Schematic  of  four  steps  used  for  generating  the  foreground  and  background  heat-map  per  texture  feature. 
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Fig.  2.  The  probability  heatmap  of  (a)  the  foreground  probability  and  (b)  the  background  probability. 
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Fig.  3.  (a)  TRUS  images  of  a  prostate  and  (b)  corresponding  the  prostate  heatmap. 
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onstrating  GGO  nodule  pathology,  a  delineation  logic  that 
distinguishes  GGO  nodule  tissue  within  the  image  from  the 
background  of  the  image,  a  texture  logic  that  extracts  a  set  of 
texture  features  from  the  image,  a  phenotype  signature  logic 
that  computes  a  phenotypic  signature  from  the  image,  a  shape 
logic  that  extracts  a  set  of  shape  features  from  the  image,  and 
a  classification  logic  that  classifies  the  GGO  nodule  tissue 
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TEXTURAL  ANALYSIS  OF  LUNG  NODULES 

CROSS  REFERENCE  TO  RELATED 

APPLICATIONS 

10001]  This  application  claims  the  benefit  of  U.S.  Provi¬ 
sional  Application  62/085.616  filed  Nov.  30.  2014. 

BACKGROUND 

[00021  Variations  of  lung  nodule  invasiveness  and  mor¬ 
phology  relate  to  prognosis  and  patient  outcomes.  One 
approach  for  diagnosing  cancer  is  histopathological  exami¬ 
nation  of  biopsy  tissue.  The  examination  may  produce  a  diag¬ 
nostic  profile  based  on  attributes  including  cell  morphology, 
cytoplasmic  changes,  cell  density,  and  cell  distribution. 
Visual  characterization  of  tumor  morphology  is,  however, 
time  consuming  and  expensive.  Visual  characterization  is 
also  subjective  and  thus  suffers  from  inter-rater  and  intra-rater 
variability.  Conventional  visual  characterization  of  lung  nod¬ 
ule  morphology  by  a  human  pathologist  may  therefore  be  less 
than  optimal  in  clinical  situations  where  timely  and  accurate 
classification  can  affect  patient  outcomes. 

[0003]  Computerized  tomography  (CT)  is  used  to  image 
nodules  in  lungs.  Chest  CT  imagery  may  be  used  to  detect  and 
diagnose  non-small  cell  lung  cancer.  However,  conventional 
approaches  have  been  challenged  when  defining  radiographic 
characteristics  that  reliably  describe  the  degree  of  invasion  of 
early  non-small  cell  lung  cancers  with  ground  glass  opacity 
(GGO).  For  example,  conventional  CT  imagery  based 
approaches  may  find  it  difficult,  if  even  possible  at  all,  to 
reliably  discriminate  nodules  caused  by  benign  fungal  infec¬ 
tions  from  non-small  cell  lung  cancer  nodules. 

|0004|  The  degree  of  invasion  of  a  lung  nodule  is  correlated 
with  prognosis.  For  example,  patients  suffering  from  mini¬ 
mally  invasive  nodules  may  have  higher  disease  free  survival 
rate  at  five  years  compared  to  patients  with  nodules  demon¬ 
strating  frank  invasion.  Since  radiologists  may  be  challenged 
to  reliably  distinguish  the  level  of  invasiveness  of  lung  nod¬ 
ules  in  situ  using  conventional  CT  approaches  in  clinically 
optimal  or  relevant  time  frames,  invasive  procedures  that  may 
be  performed  that  ultimately  result  in  a  negative  diagnosis. 
These  invasive  procedures  take  time,  cost  money,  and  put  a 
patient  at  additional  risk. 

BRIEF  DESCRIPTION  OF  THE  DRAWINGS 

|0005|  The  accompanying  drawings,  which  are  incorpo¬ 
rated  in  and  constitute  a  part  of  the  specification,  illustrate 
various  example  apparatus,  methods,  and  other  example 
embodiments  of  various  aspects  of  the  invention.  It  will  be 
appreciated  that  the  illustrated  element  boundaries  (e.g., 
boxes,  groups  of  boxes,  or  other  shapes)  in  the  figures  repre¬ 
sent  one  example  of  the  boundaries.  One  of  ordinary  skill  in 
the  art  will  appreciate  that  in  some  examples  one  element  may 
be  designed  as  multiple  elements  or  that  multiple  elements 
may  be  designed  as  one  element.  In  some  examples,  an  ele¬ 
ment  shown  as  an  internal  component  of  another  element  may 
be  implemented  as  an  external  component  and  vice  versa. 
Furthermore,  elements  may  not  be  drawn  to  scale. 

|0006]  FIG.  1  illustrates  an  example  method  of  character¬ 
izing  a  GGO  nodule  in  a  region  of  lung  tissue. 

(0007|  FIG.  2  illustrates  an  example  method  of  character¬ 
izing  a  GGO  nodule  in  a  region  of  lung  tissue. 


|0008]  FIG.  3  illustrates  an  example  method  of  distinguish¬ 
ing  invasive  tumors  from  non-invasive  tumors  in  chest  CT 
images. 

|0009|  FIG.  4  illustrates  an  example  apparatus  that  classi¬ 
fies  a  region  of  tissue  in  an  image. 

100101  FIG.  5  illustrates  an  example  computer  in  which 
example  methods  and  apparatus  described  herein  operate. 
[001 1 1  FIG.  6  illustrates  textural  features  of  a  CT  image  of 
a  granuloma  and  a  carcinoma. 

[00 1 2|  FIG.  7  is  a  boxplot  of  CT  intensities  of  a  CT  image 
of  a  minimally  invasive  GGO  nodule  and  a  frank  invasive 
GGO  nodule. 

[00 1 3]  FIG.  8  is  a  boxplot  of  CT  contrast  variance  of  a  CT 
image  of  minimally  invasive  GGO  nodule  and  frank  invasive 
GGO  nodule. 

DETAILED  DESCRIPTION 

(0014]  Variations  in  tumor  invasiveness  and  morphology 
may  be  related  to  patient  prognosis  and  outcome.  In  particu¬ 
lar,  a  GGO  nodule’s  level  of  invasion  is  strongly  correlated  to 
patient  prognosis.  Conventional  methods  of  diagnosing  can¬ 
cer  include  visual  histopathological  examination  of  a  biopsy 
to  create  a  diagnostic  profile  based  on  variations  in  tumor 
morphology.  I  lowever,  invasive  biopsy  may  not  always  be  a 
convenient  or  appropriate  method  for  assessing  GGO  nod¬ 
ules.  Invasive  biopsies  cost  money,  take  time,  and  put  a  patient 
at  additional  risk.  A  non-invasive  approach  that  provided 
improved  accuracy  compared  to  conventional  CT  based 
approaches  would  reduce  the  number  of  unnecessary  inter¬ 
ventions.  reduce  the  dependency  on  repetitive  or  higher  reso¬ 
lution  CT  exams,  offer  a  non-invasive  means  of  assessing 
response  to  targeted  therapies,  and  improve  patient  outcomes. 
Thus,  a  timely,  non-invasive  procedure  that  results  in  more 
accurate  discrimination  between  minimally  invasive  and 
frank  invasive  nodules  would  offer  reduced  risk  to  patients 
while  providing  economic  benefits  to  the  health  care  system. 
[0015]  CT  imagery  is  conventionally  used  to  differentiate 
malignant  GGO  nodules  from  other,  non-cancerous  GGO 
nodules.  Conventional  methods  of  visually  assessing  GGO 
nodule  invasiveness  based  on  CT  imagery  are  subjective  and 
yield  intra  and  inter-reviewer  variability.  In  one  example,  of  a 
group  of  baseline  CT  chest  scans,  51%  were  found  positive 
for  lung  nodules.  However,  only  12%  of  those  lung  nodules 
were  found  to  be  malignant.  The  remainder  were  determined 
to  be  granulomas  due  to  a  prior  histoplasmosis  infection. 
Conventional  CT  approaches  may  focus  exclusively  on  detec¬ 
tion  of  lung  nodules,  or  exclusively  on  diagnosing  malig¬ 
nancy  via  CT  scans.  Example  apparatus  and  methods  dis¬ 
criminate  granulomas  caused  by  fungal  infection  from 
carcinomas.  Distinguishing  fungal  infection  from  carcinoma 
facilitates  reducing  surgical  interventions  that  ultimately 
result  in  a  diagnosis  of  histoplasmosis. 

[0016]  Example  methods  and  apparatus  more  accurately 
distinguish  malignant  GGO  nodules  from  benign  nodules. 
Since  a  more  accurate  distinction  is  made,  example  apparatus 
and  methods  thus  predict  patient  outcomes  in  a  more  consis¬ 
tent  and  reproducible  manner.  Example  methods  and  appara¬ 
tus  predict  patient  outcomes  more  accurately  than  conven¬ 
tional  methods  by  employing  computerized  textural  and 
morphologic  analysis  of  lung  CT  imagery  to  distinguish 
granulomas  due  to  fungal  infection  from  malignant  tumors.  A 
GGO  nodule  may  be  segmented  from  an  image  background. 
Features  may  be  automatically  extracted  from  the  segmented 
GGO  nodule  image.  Example  methods  and  apparatus  may 
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extract  texture  features  and  shape  features  from  the  GGO 
nodule  image.  Example  methods  and  apparatus  may  also 
extract  tortuosity  features  from  the  GGO  nodule  image. 
Malignant  lung  tumors  may  induce  irregular  changes  to  ves¬ 
sel  shapes.  Example  methods  and  apparatus  detect  and  quan¬ 
tify  vessel  tortuosity  abnormalities  on  a  tumor  neighborhood. 
A  subset  of  extracted  features  may  be  selected  using  principal 
component  analysis  (PC A)  and  then  a  classification  of  the 
GGO  nodule  image  may  be  generated  using  linear  discrimi¬ 
nant  analysis  (LDA)  or  quadratic  discriminant  analysis 
(QDA). 

|0017J  Carcinomas  may  have  a  more  chaotic  cellular  archi¬ 
tecture  than  granuloma.  The  chaotic  cellular  architecture  may 
be  correlated  to  an  energy  feature  in  an  image.  The  energy 
feature  may  be  represented  as  a  texture  feature  or  a  shape 
feature.  In  some  embodiments,  the  energy  feature  is  more 
pronounced  in  a  CT  heatmap  of  a  cancerous  GGO  nodule  than 
in  a  CT  heatmap  of  a  granuloma  because  of  the  more  chaotic 
cellular  architecture  of  the  cancerous  GGO  nodule.  FIG.  6 
illustrates  this  property  of  cancerous  GGO  nodules  compared 
with  granuloma  GGO  nodules  that  were  caused  by  benign 
fungal  infections.  The  chaotic  cellular  architecture  may  also 
be  correlated  to  tortuosity  features  of  vessels  associated  with 
a  tumor  or  a  GGO  nodule. 

|0018|  FIG.  6  illustrates,  at  610,  a  CT  scan  image  of  a 
cancerous  GGO  nodule  identified  as  a  carcinoma.  FIG.  6  also 
illustrates,  at  620.  a  CT  scan  image  of  GGO  nodule  identified 
as  a  granuloma.  FIG.  6  illustrates,  at  630.  an  energy  textural 
feature  extracted  from  intensity  values  of  image  610  of  the 
cancerous  GGO  nodule.  The  eneigy  of  a  textural  feature 
measures  local  homogeneity  within  the  image,  illustrating 
how  uniform  the  texture  is.  FIG.  6  also  illustrates  a  textural 
feature  image  640  of  the  energy  extracted  from  the  image  of 
the  benign  granuloma  620.  The  energy  feature  image  630  of 
the  cancerous  GGO  nodule  displays  a  more  pronounced 
energy  feature  than  the  benign  granuloma  texture  feature 
image  640,  which  demonstrates  the  more  chaotic  cellular 
architecture  of  a  cancerous  GGO  nodule.  FIG.  6  also  illus¬ 
trates.  at  650.  a  heatmap  of  a  Gabor  feature  of  the  cancerous 
GGO  nodule.  The  Gabor  feature  represents  texture  using  a 
sinusoidal  plane  wave  modulated  Gaussian  kernel  function. 
FIG.  6  further  illustrates,  at  660.  a  heatmap  of  a  Gabor  texture 
feature  of  the  benign  granuloma. 

|0019]  Example  methods  and  apparatus  may  also  employ 
3-fold  cross  validation  where  N=46  for  training  a  classifier 
and  N  1 6  for  testing  a  classifier.  Example  methods  and  appa¬ 
ratus  may  train  a  classifier  or  test  a  classifier  with  other, 
different  numbers  of  subjects.  For  example,  a  human  patholo¬ 
gist  may  manually  delineate  and  classify  one  hundred  GGO 
nodules  for  a  training  set  and  thirty  nodules  for  a  testing  set. 
Example  methods  and  apparatus  may  classify  the  GGO  nod¬ 
ule  image  as  a  carcinoma,  adenocarcinoma,  or  as  a  granu¬ 
loma.  Example  methods  and  apparatus  may  also  classify  the 
GGO  nodule  image  as  non-invasive,  minimally  invasive,  or 
frank  invasive.  Other  classifications  may  be  employed. 

1 0020]  Example  methods  and  apparatus  thus  improve  on 
conventional  methods  by  more  accurately  distinguishing 
between  pathological  and  benign  lung  nodules. 

(002 1 1  Example  methods  and  apparatus  distinguish  granu¬ 
loma  from  carcinoma  with  an  accuracy  of  at  least  92%  area 
under  the  curve  (AUC)  when  using  texture  features  and  shape 
features  with  a  linear  discriminant  analysis  (LDA)  classifier. 
Example  methods  and  apparatus  distinguish  frank  invasive 
GGO  nodules  from  non-invasive  or  minimally  invasive  GGO 


nodules  with  an  accuracy  of  at  least  78%  AUC  when  using 
three  texture  features  selected  by  PCA  with  a  quadratic  dis¬ 
criminant  analysis  (QDA)  classifier.  In  contrast,  conventional 
approaches  using  just  Laws  feature  achieve  accuracies  of 
approximately  0.61  AUC,  while  conventional  approaches 
using  just  Gabor  features  achieve  accuracies  of  approxi¬ 
mately  0.68  AUC.  In  these  examples,  a  minimally  invasive 
GGO  nodule  is  defined  as  a  GGO  nodule  with  5  mm  or  less 
invasion,  and  a  frank  invasive  GGO  nodule  is  defined  as  a 
GGO  nodule  with  more  than  5  nun  invasion.  In  other  embodi¬ 
ments,  minimally  invasive  GGO  nodules  and  frank  invasive 
GGO  nodules  may  be  defined  on  other  dimensions. 

|0022]  By  increasing  the  accuracy  with  which  malignant 
GGO  nodules  are  distinguished  from  benign  lung  GGO  nod¬ 
ules,  example  methods  and  apparatus  produce  the  concrete, 
real-world  technical  effect  of  reducing  the  time  required  to 
evaluate  medical  imagery  while  increasing  the  accuracy  of 
the  evaluation.  Additionally,  example  apparatus  and  methods 
increase  the  probability  that  at-risk  patients  receive  timely 
treatment  tailored  to  the  particular  pathology  they  exhibit. 
Example  methods  and  apparatus  may  also  reduce  the  number 
of  invasive  procedures  needed  to  accurately  characterize 
GGO  nodules.  The  additional  technical  effect  of  reducing  the 
expenditure  of  resources  and  time  on  patients  who  are  less 
likely  to  suffer  recurrence  or  disease  progression  is  also 
achieved.  Example  methods  and  apparatus  thus  improve  on 
conventional  methods  in  a  measurable,  clinically  significant 
way. 

[0023]  Some  portions  of  the  detailed  descriptions  that  fol¬ 
low  are  presented  in  terms  of  algorithms  and  symbolic  rep¬ 
resentations  of  operations  on  data  bits  within  a  memory. 
These  algorithmic  descriptions  and  representations  are  used 
by  those  skilled  in  the  art  to  convey  the  substance  of  their 
work  to  others.  An  algorithm,  here  and  generally,  is  conceived 
to  be  a  sequence  of  operations  that  produce  a  result.  The 
operations  may  include  physical  manipulations  of  physical 
quantities.  Usually,  though  not  necessarily,  the  physical  quan¬ 
tities  take  the  form  of  electrical  or  magnetic  signals  capable  of 
being  stored,  transferred,  combined,  compared,  and  other¬ 
wise  manipulated  in  a  logic,  and  so  on.  The  physical  manipu¬ 
lations  create  a  concrete,  tangible,  useful,  real-world  result. 
[0024]  It  has  proven  convenient  at  times,  principally  for 
reasons  of  common  usage,  to  refer  to  these  signals  as  bits, 
values,  elements,  symbols,  characters,  terms,  numbers,  and 
so  on.  It  should  be  borne  in  mind,  however,  that  these  and 
similar  terms  are  to  be  associated  with  the  appropriate  physi¬ 
cal  quantities  and  are  merely  convenient  labels  applied  to 
these  quantities.  Unless  specifically  stated  otherwise,  it  is 
appreciated  that  throughout  the  description,  terms  including 
processing,  computing,  calculating,  determining,  and  so  on. 
refer  to  actions  and  processes  of  a  computer  system,  logic, 
processor,  or  similar  electronic  device  that  manipulates  and 
transforms  data  represented  as  physical  (electronic)  quanti¬ 
ties. 

|0025|  Example  methods  may  be  better  appreciated  with 
reference  to  flow  diagrams.  While  for  purposes  of  simplicity 
of  explanation,  the  illustrated  methodologies  are  shown  and 
described  as  a  series  of  blocks,  it  is  to  be  appreciated  that  the 
methodologies  are  not  limited  by  the  order  of  the  blocks,  as 
some  blocks  can  occur  in  di  fferent  orders  and/or  concurrently 
with  other  blocks  from  that  shown  and  described.  Moreover, 
less  than  all  the  illustrated  blocks  may  be  required  to  imple¬ 
ment  an  example  methodology.  Blocks  may  be  combined  or 


48 


US  2016/0155225  A 1 


3 


Jun.  2,  2016 


separated  into  multiple  components.  Furthermore,  additional 
and/or  alternative  methodologies  can  employ  additional,  not 
illustrated  blocks. 

(0026|  FIG.  1  illustrates  an  example  computerized  method 
100  for  characterizing  a  GGO  nodule  in  a  region  of  lung 
tissue.  Method  100  includes,  at  110.  accessing  an  image  of  a 
region  of  lung  tissue.  Accessing  the  image  may  include 
accessing  a  CT  image  of  the  region  of  lung  tissue.  The  CT 
image  may  be  stored,  for  example,  in  a  computer  memory  or 
may  be  provided  across  a  computer  network.  In  one  embodi¬ 
ment.  the  CT  image  is  a  1  mm  to  5  mm  thick,  no-contrast  chest 
CT  image.  In  another  embodiment,  other  images  sizes  or 
other  imaging  techniques  may  be  employed. 

[0027|  Method  100  also  includes,  at  120,  delineating  a 
GGO  nodule  in  the  image.  The  GGO  nodule  may  be  auto¬ 
matically  delineated  by  distinguishing  GGO  nodule  tissue 
within  the  image  from  the  background  of  the  image.  The 
GGO  nodule  tissue  may  be  automatically  distinguished  using 
threshold  based  segmentation,  deformable  boundary  models, 
active-appearance  models,  active  shape  models,  graph  based 
models  including  Markov  random  fields  (MRF),  min-max  cut 
approaches,  or  other  image  segmentation  approaches. 

|0028|  Method  100  also  includes,  at  130.  extracting  a  set  of 
texture  features  from  the  image  of  the  GGO  nodule.  The  set  of 
texture  features  includes  a  gray-level  statistical  feature,  a 
steerable  Gabor  feature,  a  Haralick  feature,  a  Law  feature,  a 
I  nw-Laplacian  feature,  a  local  binary  pattern  (I, BP)  feature, 
inertia,  a  correlation  feature,  a  difference  entropy  feature,  a 
contrast  inverse  moment  feature,  or  a  contrast  variance  fea¬ 
ture.  In  one  embodiment,  the  set  of  texture  features  includes 
at  least  sixty  three  texture  features.  In  another  embodiment, 
the  set  of  texture  features  includes  at  least  one  hundred  texture 
features.  For  example,  a  set  of  one  hundred  texture  features 
may  include  1 3  Haralick  features.  4  gray  features.  1 3  gradient 
features.  19  Gabor  features,  1  LBP  feature,  25  Law  features, 
or  25  Law-Laplacian  features.  In  other  embodiments,  other 
numbers  or  types  of  texture  features  may  be  extracted. 
|0029|  Method  100  also  includes,  at  140.  selecting  a  subset 
of  texture  features  from  the  set  of  texture  features.  In  one 
embodiment,  the  subset  of  texture  features  is  selected  by 
reducing  the  set  of  texture  features  using  a  PCA.  The  PC  A  of 
the  set  of  texture  features  selects  a  subset  of  texture  features 
from  the  set  of  texture  features.  The  subset  of  texture  features 
achieves  a  threshold  level  of  discriminability.  For  example, 
the  PCA  may  select  one  energy  feature  and  one  Gabor  feature 
that  are  the  most  discriminative,  based  on  a  particular  set  of 
CT  images,  for  distinguishing  carcinoma  from  granuloma. 
The  subset  of  texture  features  may  include  as  few  as  two 
texture  features.  The  level  of  discriminability  may  be  user 
adjustable.  For  example,  in  a  first  clinical  situation,  a  subset  of 
texture  features  that  achieves  0.8  AUC  accuracy  in  distin¬ 
guishing  carcinoma  from  granuloma  may  be  acceptable.  A 
feature  may  be  considered  to  have  a  desirable  level  of  dis¬ 
criminability  when  the  means  of  two  separate  classes  are 
more  than  a  threshold  distance  from  each  other,  and  where  the 
variance  of  a  class  is  less  than  a  threshold  distance,  in  com¬ 
parison  to  the  distance  between  the  means.  In  one  embodi¬ 
ment.  the  Fisher  criterion,  which  is  the  squared  difference  of 
the  means  divided  by  the  sum  of  the  variances,  may  be  used  to 
quantitatively  establish  a  desirable  level  of  discriminability. 
|0030|  FIG.  7  is  a  boxplot  of  CT  intensities  of  a  CT  image 
of  a  minimally  invasive  GGO  nodule  and  a  frank  invasive 
GGO  nodule,  hi  FIG.  7,  the  Y-axis  indicates  the  normalized 
value  of  the  average  Hounsfield  unit  within  a  nodule  in  the  CT 


image.  FIG.  7  illustrates  the  challenge  involved  with  classi¬ 
fying  GGO  nodules  using  conventional  CT  imagery 
approaches.  FIG.  7  demonstrates  that  the  CT  intensities  of  a 
minimally  invasive  GGO  nodule  and  frank  invasive  GGO 
nodule  overlap  significantly.  In  this  example,  the  median  of 
the  CT  intensity  for  the  minimally  invasive  GGO  nodule  is 
within  a  threshold  distance  from  the  median  of  the  frank 
invasive  GGO.  Thus,  an  attempt  to  classify  a  GGO  nodule 
based  on  just  CT  intensity  is  unlikely  to  be  acceptably  accu¬ 
rate. 

|003 1 1  FIG.  8  is  a  boxplot  of  CT  contrast  variance  of  a  CT 
image  of  a  minimally  invasive  GGO  nodule  and  a  frank  inva¬ 
sive  GGO  nodule.  In  FIG.  8.  the  Y-axis  indicates  the  normal¬ 
ized  value  of  the  average  Hounsfield  unit  with  a  nodule  in  the 
CT  image.  FIG.  8  demonstrates  how  texture  features,  like 
contrast  variance,  may  have  a  more  desirable  level  of  discrim¬ 
inability.  In  this  example,  the  median  of  the  CT  contrast 
variance  for  the  minimally  invasive  GGO  nodule  is  more  than 
a  threshold  distance  from  the  median  of  the  CT  contrast 
variance  for  the  frank  invasive  GGO. 

[0032]  Method  1 00  also  includes,  at  1 50.  generating  a  phe¬ 
notypic  signature  for  the  nodule.  In  one  embodiment,  the 
phenotypic  signature  is  generated  using  Fisher  criteria  rank¬ 
ing.  In  another  embodiment,  the  phenotypic  signature  is  gen¬ 
erated  using  other  techniques. 

1 0033 1  Method  100  also  includes,  at  160.  controlling  a 
computer  aided  diagnosis  (CADx)  system  to  generate  a  clas¬ 
sification  of  the  GGO  nodule  in  the  image.  The  classification 
may  be  based,  at  least  in  part,  on  the  subset  of  texture  features 
or  the  phenotypic  signature.  In  one  embodiment,  the  CADx 
system  generates  the  classification  of  the  image  of  the  GGO 
nodule  using  a  QDA  classifier.  In  another  embodiment,  the 
CADx  system  may  generate  the  classification  using  other, 
different  types  of  classifier.  The  classifier  may  be  trained  and 
tested  on  a  set  of  images  of  pre-classified  GGO  nodules.  In 
one  embodiment,  the  image  is  of  a  region  of  tissue  demon¬ 
strating  adenocarcinoma  pathology.  Controlling  the  CADx 
system  to  generate  the  classification  of  the  GGO  nodule 
based,  at  least  in  part,  on  the  subset  of  texture  features  and  the 
phenotypic  signature,  includes  classifying  the  image  of  the 
GGO  nodule  as  frank  invasive  adenocarcinoma  or  minimally 
invasive  adenocarcinoma. 

[0034]  Bxample  methods  and  apparatus  facilitate  more 
accurate  characterization  of  GGO  nodules  found  in  CT 
images  than  conventional  approaches.  Hxample  methods  and 
apparatus  thus  improve  on  conventional  methods  by  charac¬ 
terizing  GGO  nodules  as  frank  invasive,  non-invasive,  or 
minimally  invasive,  or  as  carcinomas,  adenocarcinomas,  or 
granulomas  with  greater  accuracy  and  with  less  subjective 
variability  than  conventional  methods.  Rxample  methods  and 
apparatus  therefore  facilitate  more  judicious  application  of 
biopsies  and  surgical  resection  in  a  population  undergoing  CT 
screening  for  lung  cancer. 

|0035|  Using  a  more  appropriately  determined  and  applied 
treatment  may  lead  to  less  therapeutics  being  required  for  a 
patient  or  may  lead  to  avoiding  or  delaying  a  biopsy,  a  resec¬ 
tion,  or  other  invasive  procedure.  When  regions  of  cancerous 
tissue,  including  GGO  nodules  detected  in  CT  scans,  are 
more  quickly  and  more  accurately  classified,  patients  with 
poorer  prognoses  may  receive  a  higher  proportion  of  scarce 
resources  (e.g.,  therapeutics,  physician  time  and  attention, 
hospital  beds)  while  those  with  better  prognoses  may  be 
spared  unnecessary  treatment,  which  in  turn  spares  unneces¬ 
sary  expenditures  and  resource  consumption.  Example  meth- 
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ods  and  apparatus  may  thus  have  the  real-world,  quantifiable 
effect  of  improving  patient  outcomes. 

(00361  While  FIG.  1  illustrates  various  actions  occurring  in 
serial,  it  is  to  be  appreciated  that  various  actions  illustrated  in 
FIG.  1  could  occur  substantially  in  parallel.  By  way  of  illus¬ 
tration,  a  first  process  could  delineate  a  GGO  nodule  in  a  CT 
image,  a  second  process  could  extract  texture  features  from 
the  CT  image,  and  a  third  process  could  extract  shape  features 
from  the  CT  image.  While  three  processes  are  described,  it  is 
to  be  appreciated  that  a  greater  or  lesser  number  of  processes 
could  be  employed  and  that  lightweight  processes,  regular 
processes,  threads,  and  other  approaches  could  be  employed. 
(0037]  FIG.  2  illustrates  an  example  method  200  for  char¬ 
acterizing  a  GGO  nodule  in  a  region  of  lung  tissue.  Method 
200  is  similar  to  method  100  but  includes  additional  actions. 
Method  200  includes  actions  210.  220.  230,  240.  and  250 
which  arc  similar  to  actions  110.  120.  150.  140.  and  150 
described  above  with  respect  to  method  100. 

[0038]  Method  200  also  includes,  at  260.  extracting  a  set  of 
shape  features  from  the  image  of  the  GGO  nodule.  The  set  of 
shape  features  includes  a  location  feature,  a  size  feature,  a 
width  feature,  a  height  feature,  a  depth  feature,  a  perimeter 
feature,  an  eccentricity  feature,  an  eccentricity  standard 
deviation,  a  compactness  feature,  a  roughness  feature,  an 
elongation  feature,  a  convexity  feature,  an  extend  feature,  an 
equivalent  diameter  feature,  or  a  sphericity  feature.  The  loca¬ 
tion  feature  describes  the  spatial  information  of  a  pixel  in  the 
image  of  the  GGO  nodule,  the  size  feature  describes  the 
number  of  pixels  within  the  segmented  image  of  the  GGO 
nodule,  and  the  perimeter  feature  describes  the  distance 
around  the  boundary  of  the  segmented  GGO  nodule.  The 
eccentricity  feature  describes  the  eccentricity  of  an  ellipse 
that  has  the  same  second  moments  as  the  nodule.  The  com¬ 
pactness  feature  describes  the  isoperimetric  quotient  of  the 
nodule.  The  roughness  feature  describes  the  perimeter  of  a 
lesion  in  a  slice  of  the  image  of  the  GGO  nodule  divided  by 
the  convex  perimeter  of  the  lesion.  The  elongation  feature 
describes  the  ratio  of  minor  axis  to  the  major  axis  of  the  image 
of  the  GGO  nodule,  and  the  convexity  feature  describes  the 
ratio  of  a  tumor  image  slice  to  the  convex  hull  of  the  tumor. 
The  extend  feature  describes  the  ratio  of  pixels  in  the  tumor 
region  to  pixels  in  the  total  bounding  box.  The  equivalent 
diameter  feature  describes  the  diameter  of  a  circle  having  the 
same  area  as  a  tumor  image  slice,  and  the  sphericity  feature 
describes  the  three-dimensional  compactness  of  the  nodule. 
In  one  embodiment  the  set  of  shape  features  includes  at  least 
twenty-five  shape  features.  In  another  embodiment,  the  set  of 
shape  features  may  include  other  numbers  of  shape  features, 
or  other,  different  shape  features.  A  feature  may  be  calculated 
in  three  dimensional  (3D)  space,  or  in  two  dimensional  (2D) 
space.  For  example,  width,  height,  depth,  or  sphericity  fea¬ 
tures  may  be  calculated  in  3D  space. 

(0039|  Method  200  also  includes,  at  270.  selecting  a  subset 
of  shape  features  from  the  set  of  shape  features.  In  one 
embodiment,  the  subset  of  shape  features  includes  eccentric¬ 
ity.  eccentricity  standard  deviation,  or  elongation  features.  In 
another  embodiment,  the  subset  of  shape  features  may 
include  other,  different  shape  features.  The  subset  of  shape 
features  may  be  selected  from  the  set  of  shape  features  using 
PC  A. 

(0040]  Method  200  also  includes,  at  280.  controlling  the 
CADx  system  to  generate  the  classification  of  the  image  of 
the  GGO  nodule  as  a  carcinoma  or  a  granuloma.  The  classi¬ 
fication  may  be  based,  at  least  in  part,  on  the  subset  of  texture 


features  and  the  subset  of  shape  features.  Basing  the  classifi¬ 
cation  on  both  the  subset  of  texture  features  and  the  subset  of 
shape  features  improves  on  conventional  approaches  by 
equivalent  diameter  feature,  or  a  sphericity  feature.  The  loca¬ 
tion  feature  describes  the  spatial  infomiation  of  a  pixel  in  the 
image  of  the  GGO  nodule,  the  size  feature  describes  the 
number  of  pixels  within  the  segmented  image  of  the  GGO 
nodule,  and  the  perimeter  feature  describes  the  distance 
around  the  boundary  of  the  segmented  GGO  nodule.  The 
eccentricity  feature  describes  the  eccentricity  of  an  ellipse 
that  has  the  same  second  moments  as  the  nodule.  The  com¬ 
pactness  feature  describes  the  isoperimetric  quotient  of  the 
nodule.  The  roughness  feature  describes  the  perimeter  of  a 
lesion  in  a  slice  of  the  image  of  the  GGO  nodule  divided  by 
the  convex  perimeter  of  the  lesion.  The  elongation  feature 
describes  the  ratio  of  minor  axis  to  the  major  axis  of  the  image 
of  the  GGO  nodule,  and  the  convexity  feature  describes  the 
ratio  of  a  tumor  image  slice  to  the  convex  hull  of  the  tumor. 
The  extend  feature  describes  the  ratio  of  pixels  in  the  tumor 
region  to  pixels  in  the  total  bounding  box.  The  equivalent 
diameter  feature  describes  the  diameter  of  a  circle  having  the 
same  area  as  a  tumor  image  slice,  and  the  sphericity  feature 
describes  the  three-dimensional  compactness  of  the  nodule. 
In  one  embodiment  the  set  of  shape  features  includes  at  least 
twenty-five  shape  features.  In  another  embodiment,  the  set  of 
shape  features  may  include  other  numbers  of  shape  features, 
or  other,  different  shape  features.  A  feature  may  be  calculated 
in  three  dimensional  (3D)  space,  or  in  two  dimensional  (2D) 
space.  For  example,  width,  height,  depth,  or  sphericity  fea¬ 
tures  may  be  calculated  in  3D  space. 

(0041  ]  Method  200  also  includes,  at  270.  selecting  a  subset 
of  shape  features  from  the  set  of  shape  features.  In  one 
embodiment,  the  subset  of  shape  features  includes  eccentric¬ 
ity.  eccentricity  standard  deviation,  or  elongation  features.  In 
another  embodiment,  the  subset  of  shape  features  may 
include  other,  difi’erent  shape  features.  The  subset  of  shape 
features  may  be  selected  from  the  set  of  shape  features  using 
PCA. 

[0042]  Method  200  also  includes,  at  280.  controlling  the 
CADx  system  to  generate  the  classification  of  the  image  of 
the  GGO  nodule  as  a  carcinoma  or  a  granuloma.  The  classi¬ 
fication  may  be  based,  at  least  in  part,  on  the  subset  of  texture 
features  and  the  subset  of  shape  features.  Basing  the  classifi¬ 
cation  on  both  the  subset  of  texture  features  and  the  subset  of 
shape  features  improves  on  conventional  approaches  by 
increasing  the  accuracy  with  which  the  image  of  the  GGO 
may  be  classified.  In  one  embodiment,  the  CADx  system 
generates  the  classification  of  the  image  of  the  GGO  nodule 
using  a  LDA  classifier  or  a  QDA  classifier.  In  one  embodi¬ 
ment.  an  LDA  classifier  using  a  median  textural  feature  and 
eccentricity  standard  deviation  shape  feature  achieves  an 
accuracy  of  at  least  0.92  AUC.  The  LDA  classifier  or  the  QDA 
classifier  may  be  trained  and  tested  on  a  set  of  GGO  images 
pre-classified  as  carcinoma  or  granuloma. 

|0043]  Inone  embodiment,  method  200  may  also  automati¬ 
cally  segment  vessels  associated  with  the  nodule.  Method  200 
may  identify  a  centerline  of  a  vessel  and  branching  points 
associated  with  the  vessel.  Method  200  calculates  the  torsion 
for  a  vessel  segment  using  a  distance  metric.  The  torsion  of  a 
vessel  segment  is  defined  as  1 -(Distance/Length)  where  dis¬ 
tance  is  the  Euclidean  distance  of  the  start  and  end  point  of  the 
segment,  and  where  length  is  the  number  of  voxels  along  the 
vessel  segment.  Method  200  also  extracts  the  curvature  of  a 
vessel  segment.  Curvature  at  a  voxel  of  a  vessel  segment  is 


50 


US  2016/0155225  A 1 


5 


Jun.  2,  2016 


proportional  to  the  inverse  of  an  osculating  circle’s  radius. 
The  osculating  circle  is  fitted  to  a  collection  of  three  neigh¬ 
boring  points  along  the  centerline  of  a  vessel.  For  a  plurality 
of  points  along  the  center  line  of  a  vessel,  method  200  fits  a 
circle  to  compute  the  curvature  of  a  specific  point.  Method 
200  then  computes  mean  and  standard  deviation  of  the  cur¬ 
vature  for  points  along  the  vessel. 

(0044]  Method  200  may  then  extract  a  set  of  tortuosity 
features  from  the  image  of  the  GGO  nodule.  The  tortuosity 
features  describe  vessels  associated  with  the  GGO  nodule. 
The  set  of  tortuosity  features  includes  the  mean  of  torsion  of 
a  vessel  segment,  or  the  standard  deviation  of  torsion  of  a 
vessel  segment.  The  set  of  tortuosity  features  also  includes  the 
mean  and  standard  deviation  of  the  mean  curvature  of  a  group 
of  vessel  segments.  The  set  of  tortuosity  features  also  includes 
the  mean  and  standard  deviation  of  the  standard  deviation  of 
a  vessel  segment  curvature  and  a  total  vessel  segment  length. 
In  one  embodiment,  the  set  of  tortuosity  features  includes  at 
least  seven  tortuosity  features.  In  another  embodiment,  the  set 
of  tortuosity  features  may  include  other  numbers  of  tortuosity 
features,  or  other,  different  tortuosity  features.  Method  200 
may  also  select  of  subset  of  tortuosity  features  from  the  set  of 
tortuosity  features.  Method  200  may  also  include  controlling 
the  CADx  system  to  generate  the  classification  of  the  image 
of  the  GGO  nodule  based,  at  least  in  part,  on  the  subset  of 
tortuosity  features,  the  subset  of  texture  features  and  the  sub¬ 
set  of  shape  features. 

|0045|  FIG.  3  illustrates  an  example  method  300  for  distin¬ 
guishing  invasive  tumors  from  non-invasive  tumors  in  chest 
CT  images.  Method  300  includes,  at  310  accessing  an  image 
of  a  region  of  tissue  demonstrating  cancerous  pathology.  In 
one  embodiment,  the  image  is  a  1  mm  to  5  mm  thick,  no¬ 
contrast  chest  CT  image.  In  another  embodiment,  other  image 
types  or  image  dimensions  may  be  used.  Accessing  the  image 
may  include  retrieving  electronic  data  from  a  computer 
memory,  receiving  a  computer  file  over  a  computer  network, 
or  other  computer  or  electronic  based  action. 

|0046]  Method  300  also  includes,  at  320,  segmenting  a 
tumor  in  the  image  from  the  background  of  the  image.  Seg¬ 
menting  the  tumor  in  the  image  from  the  background  of  the 
image  involves  identifying  the  portion  of  the  image  that  rep¬ 
resents  the  tumor  to  distinguish  that  portion  from  the  back¬ 
ground.  In  one  embodiment,  the  tumor  is  automatically  seg¬ 
mented  from  the  background  of  the  image.  In  another 
embodiment,  a  human  pathologist  manually  delineates  the 
tumor  from  the  background  of  the  image.  In  another  embodi¬ 
ment,  vessels  associated  with  the  tumor  are  also  segmented. 

|0047]  Method  300  also  includes,  at  330.  selecting  a  set  of 
texture  features  from  the  segmented  image.  In  one  embodi¬ 
ment.  the  set  of  texture  features  may  include  a  gray-level 
statistical  feaUire,  a  steerable  Gabor  feature,  a  Haralick  fea¬ 
ture.  a  Law  feature,  a  Law-Laplacian  feature,  an  LBP  feature, 
an  inertia  feature,  a  correlation  feature,  a  difference  entropy 
feature,  a  contrast  inverse  moment  feature,  or  a  contrast  vari¬ 
ance  feature.  In  another  embodiment,  other,  different  texture 
features  may  be  selected.  The  inertia  feature  describes  the 
contrast  or  local  intensity  variation  of  the  segmented  image. 
The  correlation  feature  describes  the  correlation  of  the  inten¬ 
sity  of  values  within  the  segmented  image.  The  difference 
entropy  feature  describes  the  disorder  of  the  difference 
between  a  pair  of  pixel  intensities  within  the  segmented 
image.  The  contrast  inverse  moment  feature  describes  the 
inhomogeneity  within  a  region  of  interest  in  the  segmented 


image.  The  contract  variance  feature  describes  the  variance  of 
the  difference  between  a  pair  of  pixel  intensities. 

[0048]  Method  300  also  includes,  at  340.  selecting  a  set  of 
shape  features  from  the  segmented  image.  Hie  set  of  shape 
features  may  include  a  location  feature,  a  size  feature,  a 
perimeter  feature,  an  eccentricity  feature,  an  eccentricity 
standard  deviation,  a  compactness  feature,  a  roughness  fea¬ 
ture,  an  elongation  feature,  a  convexity  feature,  an  equivalent 
diameter  feature,  a  radial  distance  feature,  an  area  feature,  or 
a  sphericity  feature.  The  radial  distance  feature  describes  the 
radial  distance  from  the  center  of  mass  of  the  tumor  to  a  point 
on  the  defining  contour  of  the  tumor. 

|0049J  Method  300  also  includes,  at  345.  selecting  a  set  of 
tortuosity  features  from  the  segmented  image.  The  set  of 
tortuosity  features  may  include  the  mean  of  torsion  of  a  vessel 
segment,  or  the  standard  deviation  of  torsion  of  a  vessel 
segment.  The  set  of  tortuosity  features  may  also  include  the 
mean  and  standard  deviation  of  the  mean  curvature  of  a  group 
of  vessel  segments.  The  set  of  tortuosity  features  may  also 
include  the  mean  and  standard  deviation  of  the  standard 
deviation  of  a  vessel  segment  curvature  and  a  total  vessel 
segment  length.  In  one  embodiment,  the  set  of  tortuosity 
features  includes  at  least  seven  tortuosity  features.  In  another 
embodiment,  the  set  of  tortuosity  features  may  include  other 
numbers  of  tortuosity  features,  or  other,  diff  erent  tortuosity 
features. 

(0050]  Method  300  also  includes,  at  350,  generating  a  clas¬ 
sification  for  the  tumor  based,  at  least  in  part,  on  the  set  of 
texture  features,  the  set  of  shape  features,  and  the  set  of 
tortuosity  features.  In  one  embodiment,  the  classification  is 
made  based  on  the  set  of  texture  features.  In  another  embodi¬ 
ment.  the  classification  is  based  on  the  set  of  shape  features.  In 
still  another  embodiment,  the  classification  is  based  on  a 
subset  of  the  set  of  texture  features,  a  subset  of  the  set  of  shape 
features,  and  a  subset  of  the  set  of  tortuosity  features.  The 
subset  of  the  set  of  texture  features  may  be  selected  from  the 
set  of  texture  features  using  PCA.  The  subset  of  the  set  of 
shape  features  may  be  selected  from  the  set  of  shape  features 
using  PCA.  Ihe  subset  of  the  set  of  tortuosity  features  may  be 
selected  from  the  set  of  tortuosity  features  using  PCA.  The 
subset  of  shape  features,  the  subset  of  texture  features,  or  the 
subset  of  tortuosity  features  may  be  selected  to  achieve  a 
threshold  level  of  accuracy  when  classifying  tumors.  In  one 
embodiment,  method  300  classifies  the  tumor  as  a  carcinoma 
or  a  granuloma.  In  another  embodiment,  the  tumor  is  classi¬ 
fied  as  frank  invasive,  minimally  invasive,  or  non-invasive. 
The  classification  may  be  made  by  a  CADx  system  using  a 
QDA  classifier  or  an  LDA  classifier. 

[005 1  ]  Method  300  also  includes,  at  360,  providing  a  prog¬ 
nosis  prediction  based  on  the  classification.  For  example, 
method  300  may.  at  360.  provide  a  probability  that  a  patient 
will  experience  a  lower  five  year  survival  rate  if  the  tumor  is 
classified  as  frank  invasive.  Method  300  may  alternately  pro¬ 
vide  a  probability  that  a  patient  will  experience  a  higher  five 
year  survival  rate  if  the  tumor  is  classified  as  non-invasive. 
(0052]  In  one  example,  a  method  may  be  implemented  as 
computer  executable  instructions.  Thus,  in  one  example,  a 
computer-readable  storage  medium  may  store  computer 
executable  instructions  that  if  executed  by  a  machine  (e.g., 
computer)  cause  the  machine  to  perform  methods  described 
or  claimed  herein  including  method  100,  method  200.  and 
method  300.  While  executable  instructions  associated  with 
the  listed  methods  are  described  as  being  stored  on  a  com¬ 
puter-readable  storage  medium,  it  is  to  be  appreciated  that 
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executable  instructions  associated  with  other  example  meth¬ 
ods  described  or  claimed  herein  may  also  be  stored  on  a 
computer-readable  storage  medium.  In  different  embodi¬ 
ments  the  example  methods  described  herein  may  be  trig¬ 
gered  in  different  ways.  In  one  embodiment,  a  method  may  be 
triggered  manually  by  a  user.  In  another  example,  a  method 
may  be  triggered  automatically. 

[0053]  FIG.  4  illustrates  an  example  apparatus  400  for  clas¬ 
sifying  a  region  of  tissue  in  an  image.  Apparatus  400  includes 
a  processor  4 1 0,  a  memory  420.  a  set  of  logics  440.  and  an 
interface  430  that  connects  the  processor  410.  the  memory 
420.  and  the  set  of  logics  440.  The  set  of  logics  440  includes 
an  image  acquisition  logic  441.  a  delineation  logic  443.  a 
texture  logic  445.  a  phenotype  signature  logic  446.  a  shape 
logic  447.  and  a  classification  logic  449.  In  one  embodiment, 
the  functionality  associated  with  the  set  of  logics  440  may  be 
performed,  at  least  in  part,  by  hardware  logic  components 
including,  but  not  limited  to.  field-programmable  gate  arrays 
(FPGAs),  application  specific  integrated  circuits  (ASICs), 
application  specific  standard  products  (ASSPs).  system  on  a 
chip  systems  (SOCs),  or  complex  programmable  logic 
devices  (CPLDs).  In  one  embodiment,  individual  members  of 
the  set  of  logics  440  are  implemented  as  ASICs  or  SOCs. 
[0054]  Image  acquisition  logic  441  acquires  an  image  of  a 
region  of  tissue.  'Hie  image  may  be  acquired  from,  for 
example,  a  CT  apparatus.  The  region  of  tissue  may  be  a 
section  of  tissue  demonstrating  cancerous  pathology  in  a 
patient.  The  image  of  the  region  of  tissue  may  include  an 
image  of  a  GGO  nodule.  In  one  embodiment,  the  image  is  a  1 
mm  to  5  nun  thick,  no-contrast  chest  CT  image.  Other  imag¬ 
ing  approaches  may  be  used  to  generate  and  access  the  image 
accessed  by  image  acquisition  logic  441 .  Other  image  dimen¬ 
sions  may  also  be  used. 

[0055]  Delineation  logic  443  automatically  delineates  the 
GGO  nodule  by  distinguishing  (KK)  nodule  tissue  within  the 
image  from  the  background  of  the  image.  Delineation  logic 
443  automatically  delineates  the  GGO  nodule  using  threshold 
based  segmentation,  deformable  boundary  models,  active- 
appearance  models,  active  shape  models,  graph  based  models 
including  Markov  random  fields  (MRF),  min-max  cut 
approaches,  or  other  image  segmentation  approaches. 

1 0056]  Texture  logic  445  extracts  a  set  of  texture  features 
from  the  image.  The  set  of  texture  features  may  be  extracted 
faun  the  image  of  the  delineated  GGO  nodule.  In  one 
embodiment,  the  set  of  texture  features  includes  a  gray-level 
statistical  feature,  a  steerable  Gabor  feature,  a  Haralick  fea¬ 
ture.  a  Law  feamre.  a  Law-Laplacian  feature,  an  LBP  feature, 
inertia,  correlation,  difference  entropy,  contrast  inverse 
moment,  or  contrast  variance.  The  texture  logic  445  may  also 
select  a  subset  of  texture  features  from  the  set  of  texture 
features.  Texture  logic  445  may  select  the  subset  of  texture 
features  based  on.  at  least  in  part,  a  PCA  of  the  set  of  texture 
features. 

1 0057]  Phenotype  selection  logic  446  computes  a  pheno¬ 
typic  signature  of  the  delineated  GGO  nodule  in  the  image. 
Phenotype  selection  logic  446  may  compute  the  phenotypic 
signature  using  a  Fisher  criteria  ranking. 

[0058]  Shape  logic  447  extracts  a  set  of  shape  features  from 
the  image.  Ihe  set  of  shape  features  may  include  a  location 
feature,  a  size  feature,  a  perimeter  feature,  an  eccentricity 
feature,  an  eccentricity  standard  deviation,  a  compactness 
feature,  a  roughness  feature,  an  elongation  feature,  a  convex¬ 
ity  feature,  an  equivalent  diameter  feature,  or  a  sphericity 


feature.  Shape  logic  447  also  selects  a  subset  of  shape  features 
from  the  set  of  shape  features  based,  at  least  in  part,  on  a  PCA 
of  the  set  of  shape  features. 

[0059]  Classification  logic  449  classifies  the  GGO  nodule 
tissue  based,  at  least  in  part,  on  the  set  of  texture  features,  the 
phenotypic  signature,  or  the  set  of  shape  features.  In  one 
embodiment,  classification  logic  449  logic  classifies  the 
CKiO  nodule  tissue  as  a  carcinoma  or  a  granuloma  using  an 
LDA  of  the  subset  of  texture  features  and  the  subset  of  shape 
features.  In  another  embodiment,  classification  logic  449 
classifies  the  GGO  nodule  tissue  as  minimally  invasive  or  as 
frank  invasive  using  a  QDA  of  the  subset  of  texture  features. 
In  still  another  embodiment,  classification  logic  449  may 
classify  the  GGO  nodule  tissue  using  other  analytical  tech¬ 
niques. 

[0060]  In  another  embodiment,  classification  logic  449 
may  control  a  CADx  system  to  classify  the  image  based,  at 
least  in  part,  on  the  classification.  For  example,  classification 
logic  449  may  control  a  lung  cancer  CADx  system  to  classify 
the  image  based,  at  least  in  part,  on  the  set  of  texture  features 
and  set  of  shape  features.  In  other  embodiments,  other  types 
of  CADx  systems  may  be  controlled,  including  CADx  sys¬ 
tems  for  distinguishing  GGO  nodules  among  oral  cancer, 
prostate  cancer,  colon  cancer,  brain  cancer,  and  other  diseases 
where  disease  classification  and  prognosis  prediction  may  be 
based  on  textural  or  shape  features  quantified  from  CT  images 
of  a  GGO  nodule. 

[0061]  In  one  embodiment  of  apparatus  400.  the  set  of 
logics  440  also  includes  a  tortuosity  logic.  The  tortuosity 
logic  identifies  a  vessel  associated  with  the  GGO  nodule.  The 
tortuosity  logic  identifies  the  centerline  and  a  branching  point 
of  the  vessel  associated  with  the  GGO  nodule.  The  tortuosity 
logic  computes  a  torsion  for  the  segment  of  the  vessel.  The 
tortuosity  logic  also  computes  a  curvature  of  a  voxel  of  a 
vessel  segment,  where  the  curvature  is  proportional  to  the 
inverse  of  an  osculating  circle’s  radius.  The  tortuosity  logic 
extracts  a  set  of  tortuosity  features  from  the  image.  The  set  of 
tortuosity  features  may  include  the  mean  of  torsion  of  a  vessel 
segment,  or  the  standard  deviation  of  torsion  of  a  vessel 
segment.  The  set  of  tortuosity  features  also  may  include  the 
mean  and  standard  deviation  of  the  mean  curvature  of  a  group 
of  vessel  segments.  The  set  of  tortuosity  features  also  may 
include  the  mean  and  standard  deviation  of  the  standard 
deviation  of  a  vessel  segment  curvature  and  a  total  vessel 
segment  length.  Hie  tortuosity  logic  also  selects  a  subset  of 
tortuosity  features  from  the  set  of  tortuosity  features  based,  at 
least  in  part,  on  a  PCA  of  the  set  of  tortuosity  features.  The 
subset  of  tortuosity  features  may  include  at  least  three  tortu¬ 
osity  features.  In  this  embodiment,  the  classification  logic 
449  classifies  the  GGO  nodule  tissue  based,  at  least  in  part,  on 
the  set  of  tortuosity  features,  the  set  of  texture  features,  the 
phenotypic  signature,  or  the  set  of  shape  features. 

[0062]  In  one  embodiment  of  apparatus  400.  the  set  of 
logics  440  also  includes  a  display  logic.  The  display  logic 
may  control  the  CADx  system  to  display  the  classification, 
the  texture  features,  or  the  shape  features  on  a  computer 
monitor,  a  smartphone  display,  a  tablet  display,  or  other  dis¬ 
plays.  Displaying  the  classification  or  the  features  may  also 
include  printing  the  classification  or  the  features.  The  display 
logic  may  also  control  the  CADx  to  display  an  image  of  the 
region  of  tissue  demonstrating  a  GGO  nodule.  The  image  of 
the  region  of  tissue  demonstrating  a  GGO  nodule  may  include 
a  delineated  or  segmented  representation  of  the  GGO  nodule. 
By  displaying  the  features  and  the  image  of  the  GGO  nodule. 


52 


US  2016/0155225  A 1 


7 


Jun.  2,  2016 


example  apparatus  provide  a  timely  and  intuitive  way  for  a 
human  pathologist  to  more  accurately  classify  pathologies 
demonstrated  by  a  patient,  thus  improving  on  conventional 
approaches  to  predicting  cancer  recurrence  and  disease  pro¬ 
gression. 

[0063]  FIG.  5  illustrates  an  example  computer  500  in  which 
example  methods  illustrated  herein  can  operate  and  in  which 
example  logics  may  be  implemented.  In  different  examples, 
computer  500  may  be  part  of  a  CT  system,  may  be  operably 
connectable  to  a  CT  system,  or  may  be  part  of  a  CADx 
system. 

(0064]  Computer  500  includes  a  processor  502.  a  memory 
504.  and  input/output  ports  510  operably  connected  by  a  bus 
508.  In  one  example,  computer  500  may  include  a  set  of 
logics  530  that  perform  a  method  of  characterizing  a  GGO 
nodule  in  a  region  of  lung  tissue.  Thus,  the  set  of  logics  530, 
whether  implemented  in  computer  500  as  hardware,  firm¬ 
ware,  software,  and/or  a  combination  thereof  may  provide 
means  (e.g..  hardware,  software)  for  characterizing  a  GGO 
nodule  in  a  region  of  lung  tissue.  In  different  examples,  the  set 
of  logics  530  may  be  permanently  and/or  removably  attached 
to  computer  500.  In  one  embodiment,  the  functionality  asso¬ 
ciated  with  the  set  of  logics  530  may  be  performed,  at  least  in 
part,  by  hardw  are  logic  components  including,  but  not  limited 
to,  field-programmable  gate  arrays  (FPGAs),  application  spe¬ 
cific  integrated  circuits  (ASICs),  application  specific  stan¬ 
dard  products  (ASSPs),  system  on  a  chip  systems  (SOCs),  or 
complex  programmable  logic  devices  (CPLDs).  In  one 
embodiment,  individual  members  of  the  set  of  logics  530  are 
implemented  as  ASICs  or  SOCs. 

1 0065|  Processor  502  can  be  a  variety  of  various  processors 
including  dual  microprocessor  and  other  multi-processor 
architectures.  Memory  504  can  include  volatile  memory  and/ 
or  non-volatile  memory.  A  disk  506  may  be  operably  con¬ 
nected  to  computer  500  via,  for  example,  an  input/output 
interface  (e.g..  card,  device)  518  and  an  input/output  port  510. 
Disk  506  may  include,  but  is  not  limited  to.  devices  like  a 
magnetic  disk  drive,  a  tape  drive,  a  Zip  drive,  a  flash  memory 
card,  or  a  memory  stick.  Furthermore,  disk  506  may  include 
optical  drives  like  a  CD-ROM  or  a  digital  video  ROM  drive 
(DVD  ROM).  Memory  504  can  store  processes  514  or  data 
517.  for  example.  Disk  506  or  memory  504  can  store  an 
operating  system  that  controls  and  allocates  resources  of 
computer  500. 

(0066]  Bus  508  can  be  a  single  internal  bus  interconnect 
architecture  or  other  bus  or  mesh  architectures.  While  a  single 
bus  is  illustrated,  it  is  to  be  appreciated  that  computer  500  may 
communicate  with  various  devices,  logics,  and  peripherals 
using  other  busses  that  are  not  illustrated  (e.g.,  PCIE,  SATA, 
Infiniband,  1394,  USB,  Ethernet). 

|0067|  Computer  500  may  interact  with  input/output 
devices  via  I/O  interfaces  518  and  input/output  ports  510. 
Input/output  devices  can  include,  but  are  not  limited  to,  digi¬ 
tal  whole  slide  scanners,  an  optical  microscope,  a  keyboard,  a 
microphone,  a  pointing  and  selection  device,  cameras,  video 
cards,  displays,  disk  506.  network  devices  520,  or  other 
devices.  Input/output  ports  510  can  include  but  are  not  limited 
to,  serial  ports,  parallel  ports,  or  USB  ports. 

|0068|  Computer  500  may  operate  in  a  network  environ¬ 
ment  and  thus  may  be  connected  to  network  devices  520  via 
I/O  interfaces  518  or  I/O  ports  510.  Through  the  network 
devices  520.  computer  500  may  interact  with  a  network. 
Through  the  network,  computer  500  may  be  logically  con¬ 
nected  to  remote  computers.  The  networks  with  which  com¬ 


puter  500  may  interact  include,  but  are  not  limited  to,  a  local 
area  network  (LAN),  a  wide  area  network  (WAN),  or  other 
networks. 

[00691  References  to  “one  embodiment”,  “an  embodi¬ 
ment”.  “one  example”,  and  “an  example”  indicate  that  the 
embodiment(s)  or  example(s)  so  described  may  include  a 
particular  feature,  structure,  characteristic,  property,  element, 
or  limitation,  but  that  not  every  embodiment  or  example  nec¬ 
essarily  includes  that  particular  feature,  structure,  character¬ 
istic,  property,  element  or  limitation.  Furthermore,  repeated 
use  of  the  phrase  “in  one  embodiment”  does  not  necessarily 
refer  to  the  same  embodiment,  though  it  may. 

(0070)  “Computer-readable  storage  medium”,  as  used 
herein,  refers  to  a  medium  that  stores  instructions  or  data. 
“Computer- readable  storage  medium”  does  not  refer  to 
propagated  signals.  A  computer-readable  storage  medium 
may  take  forms,  including,  but  not  limited  to,  non-volatile 
media,  and  volatile  media.  Non-volatile  media  may  include, 
for  example,  optical  disks,  magnetic  disks,  tapes,  and  other 
media.  Volatile  media  may  include,  for  example,  semicon¬ 
ductor  memories,  dynamic  memory,  and  other  media.  Com¬ 
mon  forms  of  a  computer-readable  storage  medium  may 
include,  but  are  not  limited  to.  a  floppy  disk,  a  flexible  disk,  a 
hard  disk,  a  magnetic  tape,  other  magnetic  medium,  an  appli¬ 
cation  specific  integrated  circuit  (ASIC),  a  compact  disk 
(CD),  other  optical  medium,  a  random  access  memory 
(RAM),  a  read  only  memory  (ROM),  a  memory  chip  or  card, 
a  memory  stick,  and  other  media  from  which  a  computer,  a 
processor  or  other  electronic  device  can  read. 

(0071  ]  “Logic”,  as  used  herein,  includes  but  is  not  limited 
to  hardware,  firmware,  software  in  execution  on  a  machine,  or 
combinations  of  each  to  perform  a  function(s)  or  an  action(s), 
or  to  cause  a  function  or  action  from  another  logic,  method,  or 
system.  Logic  may  include  a  software  controlled  micropro¬ 
cessor.  a  discrete  logic  (e.g..  ASIC),  an  analog  circuit,  a  digi¬ 
tal  circuit,  a  programmed  logic  device,  a  memory  device 
containing  instructions,  and  other  physical  devices.  Logic 
may  include  one  or  more  gates,  combinations  of  gates,  or 
other  circuit  components.  Where  multiple  logical  logics  are 
described,  it  may  be  possible  to  incorporate  the  multiple 
logical  logics  into  one  physical  logic.  Similarly,  where  a 
single  logical  logic  is  described,  it  may  be  possible  to  distrib¬ 
ute  that  single  logical  logic  between  multiple  physical  logics. 
|0072|  To  the  extent  that  the  term  “includes”  or  “including” 
is  employed  in  the  detailed  description  or  the  claims,  it  is 
intended  to  be  inclusive  in  a  manner  similar  to  the  term 
“comprising”  as  that  term  is  interpreted  when  employed  as  a 
transitional  word  in  a  claim. 

(0073J  Throughout  this  specification  and  the  claims  that 
follow,  unless  the  context  requires  otherwise,  the  words 
‘comprise’  and  ‘include’  and  variations  such  as  ‘comprising’ 
and  ‘including’  will  be  understood  to  be  terms  of  inclusion 
and  not  exclusion.  For  example,  when  such  terms  are  used  to 
refer  to  a  stated  integer  or  group  of  integers,  such  terms  do  not 
imply  the  exclusion  of  any  other  integer  or  group  of  integers. 
(0074|  To  the  extent  that  the  term  “or”  is  employed  in  the 
detailed  description  or  claims  (e.g.,  A  or  B)  it  is  intended  to 
mean  “A  or  B  or  both”.  When  the  applicants  intend  to  indicate 
“only  A  or  B  but  not  both”  then  the  term  “only  A  or  B  but  not 
both”  will  be  employed.  Thus,  use  of  the  term  “or”  herein  is 
the  inclusive,  and  not  the  exclusive  use.  See.  Bryan  A.  Gamer. 
A  Dictionary  of  Modem  Legal  Usage  624  (2d.  Ed.  1995). 
[00751  While  example  systems,  methods,  and  other 
embodiments  have  been  illustrated  by  describing  examples. 
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and  while  the  examples  have  been  described  in  considerable 
detail,  it  is  not  the  intention  of  the  applicants  to  restrict  or  in 
any  way  limit  the  scope  of  the  appended  claims  to  such  detail. 
It  is,  of  course,  not  possible  to  describe  every  conceivable 
combination  of  components  or  methodologies  for  purposes  of 
describing  the  systems,  methods,  and  other  embodiments 
described  herein.  Therefore,  the  invention  is  not  limited  to  the 
specific  details,  the  representative  apparatus,  and  illustrative 
examples  shown  and  described.  Thus,  this  application  is 
intended  to  embrace  alterations,  modifications,  and  variations 
that  fall  w  ithin  the  scope  of  the  appended  claims. 

What  is  claimed  is: 

1.  A  non-transitory  computer-readable  storage  medium 
storing  computer  executable  instructions  that  when  executed 
by  a  computer  control  the  computer  to  perform  a  method  for 
characterizing  a  ground  glass  (GGO)  nodule  in  a  region  of 
lung  tissue,  the  method  comprising: 

accessing  an  image  of  a  region  of  lung  tissue: 

delineating  a  GGO  nodule  in  the  image: 

extracting  a  set  of  texture  features  from  the  GGO  nodule: 

selecting  a  subset  of  texture  features  from  the  set  of  texture 
features: 

extracting  a  set  of  shape  features  from  the  GGO  nodule: 

selecting  a  subset  of  shape  features  from  the  set  of  shape 
features: 

generating  a  phenotypic  signature  for  the  nodule:  and 

controlling  a  computer  aided  diagnosis  (CADx)  system  to 
generate  a  classification  of  the  GGO  nodule  in  the  image 
based,  at  least  in  part,  on  the  subset  of  texture  features, 
the  subset  of  shape  features,  or  the  phenotypic  signature. 

2.  The  non-transitory  computer-readable  storage  medium 
of  claim  1.  where  accessing  the  image  of  the  region  of  lung 
tissue  includes  accessing  a  computed  tomography  (CT) 
image  of  the  region  of  lung  tissue,  where  the  CT  image  is  a 
no-contrast  chest  CT  image. 

3.  The  non-transitory  computer- readable  storage  medium 
of  claim  1.  the  method  comprising  automatically  delineating 
the  GGO  nodule  by  distinguishing  GGO  nodule  tissue  in  the 
image  from  the  background  of  the  image. 

4.  The  non-transitory  computer-readable  storage  medium 
of  claim  1.  where  the  set  of  texture  features  includes  at  least 
sixty  three  texture  features. 

5.  The  non-transitory  computer-readable  storage  medium 
of  claim  1.  where  the  set  of  texture  features  includes  a  gray- 
level  statistical  feature,  a  steerable  Gabor  feature,  a  I  laralick 
feature,  a  Law  feature,  a  Law-Laplacian  feature,  an  LBP 
feature,  an  inertia  feature,  a  correlation  feature,  a  difference 
entropy  feature,  a  contrast  inverse  moment  feature,  a  gradient 
feature,  or  a  contrast  variance  feature. 

6.  The  non-transitory  computer-readable  storage  medium 
of  claim  1.  where  selecting  the  subset  of  texture  features  from 
the  set  of  texture  features  includes  reducing  the  set  of  texture 
features  using  principal  component  analysis  (PCA). 

7.  The  non-transitory  computer-readable  storage  medium 
of  claim  1.  where  the  phenotypic  signature  is  generated  using 
Fisher  criteria  ranking. 

8.  The  non-transitory  computer-readable  storage  medium 
of  claim  1,  where  the  CADx  system  generates  the  classifica¬ 
tion  of  the  image  of  the  GGO  nodule  using  a  quadratic  dis¬ 
criminant  analysis  (QDA)  classifier. 

9.  The  non-transitory  computer-readable  storage  medium 
of  claim  1.  where  the  image  is  of  a  region  of  adenocarcinoma 
tissue,  and  where  controlling  the  CADx  system  to  generate 
the  classification  of  the  image  of  the  GGO  nodule  based,  at 


least  in  part,  on  the  subset  of  texture  features  and  the  pheno¬ 
typic  signature,  includes  classifying  the  image  of  the  GGO 
nodule  as  frank  invasive  adenocarcinoma  or  minimally  inva¬ 
sive  adenocarcinoma. 

1 0.  The  non-transitory  computer-readable  storage  medium 
of  claim  1.  where  the  set  of  shape  features  includes  a  location 
feature,  a  size  feature,  a  width  feature,  a  height  feature,  a 
depth  feature,  a  radial  distance  feature,  a  perimeter  feature,  an 
eccentricity  feature,  an  eccentricity  standard  deviation,  a 
compactness  feature,  a  roughness  feature,  an  elongation  fea¬ 
ture,  a  convexity  feature,  an  equivalent  diameter  feature,  or  a 
sphericity  feature. 

1 1 .  The  non-transitory  computer-readable  storage  medium 
of  claim  10.  where  the  subset  of  shape  features  includes  an 
eccentricity  feature,  an  eccentricity  standard  deviation  fea¬ 
ture,  or  an  elongation  feature. 

1 2.  The  non-transitory  computer-readable  storage  medium 
of  claim  II.  the  method  comprising  controlling  the  CADx 
system  to  generate  the  classification  of  the  image  of  the  GGO 
nodule  as  a  carcinoma  or  a  granuloma  based,  at  least  in  part, 
on  the  subset  of  texture  features  and  the  subset  of  shape 
features. 

1 3.  The  non-transitory  computer-readable  storage  medium 
of  claim  12.  the  method  further  comprising: 

segmenting  a  vessel  associated  with  the  image  of  the  GGO 
nodule  into  a  plurality  of  vessel  segments: 

computing  a  torsion  for  a  vessel  segment; 

computing  a  curvature  for  the  vessel  segment,  where  the 
curvature  is  proportional  to  the  inverse  of  the  radius  of  an 
osculating  circle; 

selecting  a  set  of  tortuosity  features  from  the  image  of  the 
GGO  nodule,  where  the  set  of  tortuosity  features 
includes  a  mean  of  torsion  of  the  vessel  segment,  a 
standard  deviation  of  torsion  of  the  vessel  segment,  a 
mean  of  the  mean  curvature  of  the  plurality  of  vessel 
segments,  a  standard  deviation  of  the  mean  curvature  of 
the  plurality  of  vessel  segments,  a  mean  of  the  standard 
deviation  of  the  curvature  of  the  vessel  segment,  a  stan¬ 
dard  deviation  of  the  standard  deviation  of  the  curvature 
of  the  vessel  segment,  a  total  length  of  a  vessel  segment, 
or  a  total  length  of  the  plurality  of  vessel  segments; 

selecting  a  subset  of  tortuosity  features  from  the  set  of 
tortuosity  features  using  a  PCA  of  the  set  of  tortuosity 
features;  and 

controlling  a  computer  aided  diagnosis  (CADx)  system  to 
generate  a  classification  of  the  GGO  nodule  in  the  image 
based,  at  least  in  part,  on  the  subset  of  tortuosity  features, 
the  subset  of  texture  features,  the  subset  of  shape  fea¬ 
tures.  or  the  phenotypic  signature. 

1 4.  The  non-transitory  computer-readable  storage  medium 
of  claim  1,  where  the  CADx  system  generates  the  classifica¬ 
tion  of  the  image  of  the  GGO  nodule  using  a  linear  discrimi¬ 
nant  analysis  (LDA)  classifier  or  a  quadratic  discriminant 
analysis  (QDA)  classifier. 

15.  The  non-transitory  computer-readable  storage  medium 
of  claim  14.  where  the  LDA  classifier  classifies  the  image  of 
the  GGO  nodule  with  an  accuracy  of  at  least  0.92  area  under 
the  curve  (AUC). 

16.  A  method  for  distinguishing  tumors  in  a  medical  image, 
the  method  comprising: 

accessing  an  image  of  a  region  of  tissue  demonstrating 
cancerous  pathology; 

segmenting  a  tumor  in  the  image  from  the  background  of 
the  image; 
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selecting  a  set  of  texture  features  from  the  segmented 
image; 

selecting  a  set  of  shape  features  from  the  segmented  image; 

generating  a  classification  for  the  tumor  based,  at  least  in 
part,  on  the  set  of  texture  features  and  the  set  of  shape 
features;  and 

providing  a  prognosis  prediction  based  on  the  classifica¬ 
tion. 

17.  An  apparatus  for  classifying  a  region  of  tissue  in  an 
image,  comprising: 

a  processor; 

a  memory  ; 

an  input/output  interface; 

a  set  of  logics;  and 

an  interface  to  connect  the  processor,  the  memory,  the 
input/output  interface  and  the  set  of  logics,  where  the  set 
of  logics  includes: 

an  image  acquisition  logic  that  acquires  an  image  of  a 
region  of  tissue  demonstrating  ground  glass  (GGO) 
nodule  pathology; 

a  delineation  logic  that  distinguishes  GGO  nodule  tissue 
in  the  image  from  the  background  of  the  image; 
a  texture  logic  that  extracts  a  set  of  texture  features  from 
the  image; 

a  phenotype  signature  logic  that  computes  a  phenotypic 
signature  from  the  image; 

a  shape  logic  that  extracts  a  set  of  shape  features  from  the 
image;  and 

a  classification  logic  that  classifies  the  GGO  nodule 
tissue  based,  at  least  in  part,  on  the  set  of  texture 
features,  the  phenotypic  signature,  or  the  set  of  shape 
features. 

18.  The  apparatus  of  claim  17,  where  the  set  of  texture 
features  includes  a  gray-level  statistical  feature,  a  steerable 
Gabor  feature,  a  Haralick  feature,  a  Law  feature,  a  Law- 
Laplacian  feature,  a  gradient  feature,  a  local  binary  pattern 
(LBP)  feature,  an  inertia  feature,  a  correlation  feature,  a  dif¬ 
ference  entropy  feature,  a  contrast  inverse  moment  feature,  or 
a  contrast  variance  feature,  and  where  the  texture  logic  selects 


a  subset  of  texture  features  from  the  set  of  texture  features 
based  on.  at  least  in  part,  a  principal  component  analysis 
(PCA)  of  the  set  of  texture  features. 

19.  The  apparatus  of  claim  18.  where  the  set  of  shape 
features  includes  a  location  feature,  a  size  feature,  a  width 
feature,  a  height  feature,  a  depth  feature,  a  radial  distance 
feature,  a  perimeter  feature,  an  eccentricity  feature,  an  eccen¬ 
tricity  standard  deviation  feature,  a  compactness  feature,  a 
roughness  feature,  an  elongation  feature,  a  convexity  feature, 
an  equivalent  diameter  feature,  or  a  sphericity  feature,  and 
where  the  shape  logic  selects  a  subset  of  shape  features  from 
the  set  of  shape  features  based  on.  at  least  in  part,  a  PCA  of  the 
set  of  shape  features. 

20.  The  apparatus  of  claim  19. 

where  the  classification  logic  classifies  the  GGO  nodule 
tissue  as  a  carcinoma  or  a  granuloma  using  a  linear 
discriminant  analysis  of  the  subset  of  texture  features 
and  the  subset  of  shape  features,  or 

where  the  classification  logic  classifies  the  GGO  nodule 
tissue  as  minimally  invasive  or  as  frank  invasive  using  a 
quadratic  discriminant  analysis  of  the  subset  of  texture 
features. 

2 1 .  The  apparatus  of  claim  20.  the  set  of  logics  comprising 
a  tortuosity  logic  that  extracts  a  set  of  tortuosity  features  from 
the  image,  the  set  of  tortuosity  features  including  a  mean  of 
torsion  of  the  vessel  segment,  the  standard  deviation  of  tor¬ 
sion  of  the  vessel  segment,  a  mean  of  the  mean  curvature  of 
the  plurality  of  vessel  segments,  a  standard  deviation  of  the 
mean  curvature  of  the  plurality  of  vessel  segments,  a  mean  of 
the  standard  deviation  of  the  curvature  of  the  vessel  segment, 
a  standard  deviation  of  the  standard  deviation  of  the  curvature 
of  the  vessel  segment,  a  total  length  of  a  vessel  segment,  ora 
total  length  of  the  plurality  of  vessel  segments. 

22.  The  apparatus  of  claim  21.  where  the  classification 
logic  classifies  the  GGO  nodule  tissue  based,  at  least  in  part, 
on  the  set  of  tortuosity  features,  the  set  of  texture  features,  the 
phenotypic  signature,  or  the  set  of  shape  features. 
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